Changelog

All notable Changes to the Julia package Manopt.jl are documented in this file. The file was started with Version 0.4.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

0.6.7 (September 9, 2026)

Added

• the new internal function additional_callbacks(::Type{<:AbstractManoptSolverState}) was added to declare the callback hooks instead of manually overwriting provided_callbacks. (#643)
• A BarzilaiBorweinStepsize as a standalone stepsize instead of only being available within the NonmonotoneLinesearchStepsize. (#641)
• a benchmark suite in benchmark/, written with BenchmarkTools.jl and run with AirspeedVelocity.jl, which on a pull request labeled benchmark compares it against master. It starts with two problems, the Riemannian mean on the sphere, benchmarked with gradient_descent and quasi_Newton, and the Riemannian median on hyperbolic space, benchmarked with cyclic_proximal_point. (#640)
• introduce a StepsizeInitialGuess that allows to use a Stepsize as initial guess of a line search. (#641)
• Runic.jl is now also used to check code formatting in the .qmd and .md files of the repository (#643)
• a keyword γ for interior_point_Newton, the initial value of its centrality condition.
• allocating get_linear_operator(M, neo, p, B) and get_vector_field(M, neo, p, B) for the coordinates surrogate of the normal equations. (#643)
• DebugProximalParameter and RecordProximalParameter can now also be used with proximal_point. (#643)

Changed

• the changelog check in CI is skipped on pull requests labeled no changelog necessary; dependabot adds this label to its pull requests automatically. (#643)
• since has_converged exists, the status reports on REPL now use this to indicate whether an algorithm has converged. (#643)
• the field X_old of the MomentumGradientRule is now called η_old, since it stores the accumulated momentum direction and not the last gradient. (#643)
• the backtracking of the proximal_gradient_method no longer constructs a ProximalGradientMethodState in every call, but uses two working points of its step size; the internal _pgm_proximal_step! now takes the sub problem and sub state instead of a whole state. (#643)
• the modes of RecordTime and DebugTime are now capitalized consistently, that is :Cumulative, :Iterative and :Total. (#643)
• stochastic_gradient_descent with order_type=:FixedRandom now draws a new permutation at the start of every epoch. (#643)
• the TrustRegionsState fields Z, HZ and f_proposal were removed, since they were never read; the Cauchy point is stored in Y. (#643)
• ProximalPointState is an AbstractManoptSolverState, since it stores no gradient, and it provides get_iterate and set_iterate!. (#643)

Fixed

The following fixes were reported by an AI assisted code review. Each single point was still carefully checked, and committed by hand. Most of them are minor fixes and allowing several areas of Manopt.jl to also work on decorators and other edge cases. Only very few of the fixes are actually bug fixes, e.g. the line search direction in the interior point Newton was slightly wrong. They are still all listed here in detail in case (a) someone else's code breaks or (b) it was not done carefully enough – to then avoid these approaches in the future. <the following list stems from (#643) as some changes above as well. This is the assisted review PR.

• adaptive_regularization_with_cubics now also runs with a closed-form sub solver; setting the iterate of a ClosedFormSubSolverState is a no-op instead of an error.
• adaptive_regularization_with_cubics now wraps an allocating closed-form sub solver.
• adaptive_regularization_with_cubics now passes sub_kwargs on to the decorate_state! of its sub state.
• adaptive_regularization_with_cubics no longer evaluates the gradient a second time per iteration, since the state already holds it.
• get_cost_function(::AdaptiveRegularizationWithCubicsModelObjective, true) now returns the wrapped objective's cost, like its gradient counterpart.
• AdaptiveWNGradient fixed its reference gradient norm to the initial one, so its adaptive mode now works.
• AdaptiveWNGradient, BarzilaiBorwein and NonmonotoneLinesearch now reset their internal state when a solver is initialized, so reusing such a step size object no longer continues from the previous run.
• AffineCovariantStepsize now solves the simplified Newton system with the sub problem's own evaluation type, so it works together with sub_state = InplaceEvaluation().
• alternating_gradient_descent now uses its retraction_method.
• the step size of alternating_gradient_descent evaluates only the component gradient of the current block instead of the full gradient.
• augmented_Lagrangian_method!(M, f, grad_f, p; …) no longer decorates its objective twice, so count= and cache= work for the in-place variant and an already decorated objective is accepted.
• augmented_Lagrangian_method and exact_penalty_method can now be run with a closed form sub solver.
• augmented_Lagrangian_method now measures the penalty with the multipliers the sub problem was solved with, as its own formula states, instead of with the ones just updated.
• the default stopping_criterion of AugmentedLagrangianMethodState now ends in StopWhenStepsizeLess(1.0e-10), matching augmented_Lagrangian_method.
• BarzilaiBorweinStepsize defaults max_stepsize to 1.0 on manifolds with infinite injectivity radius.
• the first BarzilaiBorwein step is now the documented maximal step size instead of a quotient of transport roundoff.
• NonmonotoneLinesearch now defaults bb_max_stepsize to the same bound as BarzilaiBorweinStepsize, that is 0.9 of the injectivity radius, instead of the constant 1e3 that let its first step leave the injectivity radius.
• cache=(:LRU, [:ProximalMap]) now also caches the proximal map without an index, the one proximal_point and proximal_gradient_method call.
• the generalized Cauchy direction no longer exceeds the isotropic maximal step size on a product manifold with a box factor.
• ChambollePock now defaults to the variant matching the operator that was provided; requesting a variant without its operator errors with an explanation.
• ChambollePock now accepts and forwards the documented inverse_retraction_method_dual and vector_transport_method_dual keywords, which were previously warned about and dropped.
• ChambollePockState accepts integer step sizes, acceleration and relaxation and promotes them.
• the error= keyword of check_gradient and check_Hessian now also applies to their check_vector step.
• check_Hessian now hands its sampling keywords to the gradient check it runs first.
• cma_es! now leaves its result (the best visited point) in the input point, which before ended at the final mean.
• cma_es now uses the fitness-sorted samples in its covariance matrix update, cf. Eq. (47) of arXiv:1604.00772.
• cma_es now uses Hansen's 1/(21n^2) term in its approximation of the expected norm of a standard normal vector.
• cma_es(M, mco, p) now also accepts an objective, like cma_es! always did, and both work on manifolds whose points are numbers, for example Circle().
• cma_es now accepts tol_fun and tol_x of different types, so switching one of them off with tol_fun = 0 works.
• conjugate_residual now uses its initial vector X — it was ignored, making runs nondeterministic — and conjugate_residual! works in place of X.
• ConjugateGradientDescentState can now be constructed without specifying a stepsize.
• the bundle cap of convex_bundle_method is also enforced when the oldest entry is the last serious iterate.
• convex_bundle_method now uses its retraction_method for the candidates of both the domain backtracking and the null step.
• the domain backtracking of convex_bundle_method no longer contracts the step when the retracted point is as far from the iterate as the tangent vector is long up to rounding.
• ConvexBundleMethodState no longer errors when only one of k_min and k_max is provided.
• ConvexBundleMethodState can be built and run on its own again.
• count=[:ProximalMap] now works for a ManifoldProximalMapObjective built from a single proximal map.
• the constraint gradient accessors of a ConstrainedManifoldObjective now default to the power representation the constraint function stores, instead of always assuming NestedPowerRepresentation(); an objective built with an ArrayPowerRepresentation errored before.
• the cached constraint gradients no longer fail when the objective has only one kind of constraint.
• a range or mask index on cached constraint gradients now returns copies, so mutating the result no longer changes the cache.
• a Vector{Bool} mask is read as a mask, like a BitVector, when selecting constraints.
• the automatic inference of the number of constraints no longer calls in-place constraint functions in allocating form; augmented_Lagrangian_method and exact_penalty_method now report the missing count instead of failing inside the user's function.
• CubicBracketingLinesearch, HagerZhangLinesearch and Nesterov now promote mixed numeric types in their keyword arguments.
• CubicBracketingLinesearch and HagerZhangLinesearch now also run on Float32 problems, where the step size and the cost have different types.
• CubicBracketingLinesearch now uses the gradient= keyword it is given and no longer reads the state field X directly, so it works for any state implementing the documented interface.
• the :Random evaluation order of cyclic_proximal_point now reshuffles every cycle; its order values are unified to :Linear, :FixedRandom, and :Random, and validated.
• the default prefixes of DebugEntryChange and DebugIfEntry name the field instead of printing a literal $f.
• DebugEvery and RecordEvery with a frequency of one now also show and record the sub solver of the first iteration.
• DebugStepsize now respects at_init for convex_bundle_method.
• debug=[:IterativeTime] now really resets the timer each iteration; the factory built it with a :Iterative mode that the functor never matched.
• a (:Stop, prefix) tuple in debug= now lands in the :Stop entry, so it prints the reason at the end instead of the prefix in every iteration.
• a debug frequency given at the top level of debug= now uses the activation offset of the entry it wraps, so a :BeforeIteration entry no longer shows the sub solver run of the previous iteration.
• decorate_state! now accepts debug/record dictionaries with a concrete action value type, such as Dict(:Stop => DebugStoppingCriterion()).
• DecreasingStepsize in :absolute mode now uses the gradient= keyword it is given, like ConstantStepsize.
• default_vector_norm(::Euclidean, p, X) returned the norm of p instead of X.
• difference_of_convex_algorithm now forwards a gradient=, also when the objective carries it.
• the closed-form difference_of_convex_algorithm now leaves the gradient of f in the state.
• difference_of_convex_proximal_point now defaults to the point-type-aware default_inverse_retraction_method(M, typeof(p)).
• difference_of_convex_proximal_point now leaves the gradient of the objective in the state's X.
• difference_of_convex_proximal_point now tells its step size which direction it steps along.
• since we introduced the differential in the first order objectives, they were not fully supported in all places. This was now fixed and unified.
• DistanceOverGradientsStepsize now promotes integer keyword arguments to floats instead of erroring on its first call.
• DomainBackTrackingStepsize and NullStepBackTrackingStepsize now start at their initial_stepsize.
• EmbeddedManifoldObjective now converts the constraint Hessians to Riemannian ones instead of returning the Euclidean ones.
• the allocating exact_penalty_method forwarded the equality-constraint count as the inequality count.
• exact_penalty_method now spells its sub-solver callbacks :BeforeSubsolver and :Subsolver, like all other solvers.
• exact_penalty_method now bounds the memory of its default quasi-Newton sub solver by the manifold dimension, as augmented_Lagrangian_method already did.
• ExactPenaltyMethodState now defaults to the same stopping criterion as exact_penalty_method.
• the storage wrappers of FletcherReevesCoefficient, LiuStoreyCoefficient and PolakRibiereCoefficient now return 0 on their first call, like the other coefficient rules.
• Frank_Wolfe_method now also fires its advertised :BeforeSubsolver, :Subsolver and :Stepsize callbacks when a closed-form sub solver is used.
• the default Frank-Wolfe sub objective is no longer converted from the embedding, since it is Riemannian by construction.
• an allocating Jacobian is wrapped for every FunctionVectorialType, so a NestedReplacingPowerRepresentation no longer fails at construction.
• get_constraints now works for any constrained objective, not only embedded ones.
• get_constraints now uses the :Constraints cache, which before was allocated but never read nor written.
• get_count(::StopWhenAny, :Iterations) no longer discards a sub-criterion that stopped at iteration 0.
• get_feasibility_status works again; its keyword defaults called the removed plural constraint accessors.
• the problem-level get_grad_inequality_constraint! now has the documented j = : default and forwards a range.
• get_gradient_function of a sub objective now returns the gradient of the sub objective itself, as its adaptive regularization variant already did.
• the uncached fallback of get_gradients! now delegates in place, and get_cost_and_gradient! for a SimpleManifoldCachedObjective writes the caller's buffer on a cache hit.
• get_hess_inequality_constraint! now returns its output buffer instead of the direction when there are no inequality constraints.
• get_initial_stepsize(amp, ams, vars...; kwargs...) now really accepts the documented keyword and positional arguments, and forwards them through a decorated state.
• get_iterate is defined for LanczosState, so recording and debugging the sub solver iterate works.
• get_linear_operator(!) and get_vector_field(!) now pass decorated objectives through, so conjugate_residual accepts count= and cache= and actually uses the cache.
• the get_objective_* accessors of a sub objective now pass through decorators.
• gradient_sampling now also works for number-typed points, like gradient_descent.
• the default stopping criterion of gradient_sampling now tests the convex hull direction instead of the plain gradient.
• the documented GradientSamplingState(M) constructor works again; its convex_hull_coeffs default referred to the static parameter R, which is not bound while keyword defaults are evaluated.
• GradientSamplingState accepts real keywords of different types and promotes them.
• HagerZhangInitialGuess now uses the retraction_method of the surrounding HagerZhangLinesearch for its quadratic step.
• HybridCoefficient now updates the storage of each of its inner coefficients at the end of a conjugate gradient step, so a rule that uses the previous search direction, like DaiYuanCoefficient or HestenesStiefelCoefficient, no longer computes its coefficient from the direction of the step before.
• initialize_solver! for a ProximalBundleMethodState now also resets lin_errors and approx_errors, so a state can be reused for a second solve!.
• initialize_solver! of LanczosState clears the solution, so a reused state does not return the previous one.
• interior_point_Newton assembled its line-search gradient with the μ- and λ-components swapped, breaking problems with both constraint types.
• interior_point_Newton!(M, f, grad_f, Hess_f, p; …) no longer decorates its objective twice, so count= and cache= now work for the in-place variant.
• interior_point_Newton now runs with a closed form sub solver.
• interior_point_Newton(M, cmo) now has the documented default start point.
• interior_point_Newton now applies objective_type=:Euclidean to its KKT sub problem and line search objective as well.
• interior_point_Newton now performs its line search along the Newton direction it steps along.
• is_Hessian_symmetric now passes its remaining keyword arguments on to isapprox, as documented.
• the Lanczos first-order-progress criterion now uses the correct model gradient norm.
• LevenbergMarquardt now defaults to a coordinate normal-system sub solver on manifolds with box constraints, where the default sub solver errored before.
• the allocating LevenbergMarquardt now honours jacobian_tangent_basis like its in-place variant, instead of silently ignoring it.
• LevenbergMarquardt now also clamps the damping term to damping_term_max when a step is rejected for being too long.
• LevenbergMarquardt with use_unified_basis=true now defaults its sub_state to a CoordinatesNormalSystemState.
• the coordinate Levenberg-Marquardt surrogate now converts tangent vectors in its stored basis instead of the manifold default.
• the basis variants of the Levenberg-Marquardt surrogate now use the cached residuals like their tangent vector counterparts, which also saves two residual evaluations per step.
• LevenbergMarquardt evaluates the cost once per iteration and through the decorated objective, so count= and cache= apply to it.
• LevenbergMarquardtState no longer needs its Jacobian cache pre-shaped and reports an invalid damping_reduction_factor with an ArgumentError.
• linesearch_backtrack no longer errors when called without gradient information; it then backtracks on a plain decrease condition.
• the LineSearches.jl step size now hands its retraction to the initial guess.
• mesh_adaptive_direct_search now moves the poll base point to the current iterate before the search, so the search is handed a direction tangent at that iterate and the iterates stay on the manifold.
• mesh_adaptive_direct_search now defaults its retraction and vector transport with the point type instead of the number type.
• mesh_adaptive_direct_search now regenerates its random vector b_l whenever the mesh size changes, instead of reusing the one of a finer mesh.
• MeshAdaptiveDirectSearchState now defaults scale_mesh and its stopping criterion to the same values as mesh_adaptive_direct_search, that is injectivity_radius(M)/4 and StopWhenPollSizeLess(1e-10).
• MomentumGradient no longer folds the step size into its stored direction, so solvers no longer apply the step twice; with momentum=0 it now reduces to plain gradient descent.
• NelderMead now honours return_objective=true.
• fix a sign in the Nesterov acceleration parameter equation.
• nondescent_direction_behavior=:step_towards_negative_gradient now steps towards the negative gradient.
• the allocating particle_swarm no longer overwrites a user supplied velocity.
• primal_dual_semismooth_Newton! now works in place of its point and is exported.
• the in-place primal_dual_semismooth_Newton! accepts a callable cost, like its allocating variant.
• primal_dual_semismooth_Newton now uses the vector_transport_method= keyword also in its ∂X₁₂ block, and expands that block in the basis at the iterate p instead of at m.
• primal_dual_semismooth_Newton now builds its ∂X₁₁ basis directions at the iterate p, where the cached basis lives, instead of at m.
• PrimalDualSemismoothNewtonState now defaults its dual variable X to zero_vector(N, n), the tangent space it is actually used in.
• PrimalDualSemismoothNewtonState is now constructed as (M, N; kwargs...) like ChambollePockState
• projected_gradient_method accepts a number as start point, for example on the Circle.
• projected_gradient_method evaluates the gradient once per iteration instead of twice.
• proximal_bundle_method and convex_bundle_method now default to the in-place subsolver when used with InplaceEvaluation.
• proximal_gradient_method now supports a function-valued sub_problem as documented, wrapping it only for an evaluation type and dispatching the closed-form proximal step.
• proximal_gradient_method now accepts a decorated objective, and its :convex backtracking strategy works together with count=/cache=.
• proximal gradient backtracking now stores the collapsed step size, so a collapse is detected.
• proximal_gradient_method now rejects an incomplete sub-solver setup up front.
• proximal_gradient_method now also warns with the threshold of DebugWarnIfStepsizeCollapsed itself, not only with the one of its backtracking step size.
• proximal_gradient_method now works with a number as start point, for example on Circle().
• proximal_gradient_method now stores the point its proximal gradient step starts from, so StopWhenGradientMappingNormLess measures the documented gradient mapping also under acceleration.
• the default proximal parameter of proximal_point is k -> 1.0, so it no longer forces integers.
• ProximalGradientMethodAcceleration now uses the inverse retraction it is configured with.
• ProximalGradientMethodBacktrackingStepsize accepts numeric keywords of different types and promotes them.
• ProximalGradientNonsmoothCost now computes the documented 1/(2λ) proximity weight instead of λ/2.
• the box quasi-Newton update now treats a deactivated initial_scale (as set by a preconditioner) as 1, instead of erroring.
• quasi_Newton and quasi_Newton! now forward evaluation= to the objective-based method, so a mutating preconditioner is no longer wrapped as allocating.
• quasi_Newton now reports that cautious_update=true is not supported on manifolds with an anisotropic maximal step size, instead of failing with a MethodError.
• the empty-memory branch of the box quasi-Newton hessian_value now divides by initial_scale like its siblings, so the two accessors agree.
• the cautious quasi-Newton matrix update now evaluates its bound at the previous iterate, as documented.
• quasi_Newton now divides the maximal step size by the Riemannian norm on anisotropic manifolds.
• quasi_Newton no longer turns the iterate into NaN, or the box constrained variant into a DimensionMismatch, when a step vanishes because the minimizer was reached before the stopping criterion fires.
• quasi_Newton! now accepts and forwards the documented nondescent_direction_behavior keyword.
• initial_scale=nothing now deactivates the initial scaling for the matrix-based quasi-Newton updates.
• the cautious quasi-Newton matrix update compares the real part of the inner product against its bound.
• the :byrd curvature test in the limited-memory quasi-Newton update now squares the gradient norm.
• the limited memory quasi-Newton direction now applies the preconditioner also while its memory is empty or all stored pairs are discarded.
• QuasiNewtonState now activates the default initial scaling when no preconditioner is given, matching quasi_Newton.
• RecordFactory no longer throws for an entry vector that contains no recording action.
• RecordIterate(T::DataType) now builds RecordIterate{T} as documented, instead of a broken RecordIterate{DataType}.
• RecordTime(; mode=:Total) now resets its recorded values when the solver state is re-initialized.
• reflect! now defaults its inverse_retraction_method with the point type, like reflect.
• ReturnSolverState now passes set_parameter! and get_parameter on, so a callback can adjust the stopping criterion with return_state=true as well.
• set_iterate! for a ProximalGradientMethodState no longer rebinds the state's iterate to the caller's point.
• set_iterate! of DouglasRachfordState now also updates the reflection point, so it takes effect.
• set_iterate! of both bundle states now also sets the iterate, so the value survives the next solve!.
• set_parameter! on a decorated objective now passes the update on to the objective it wraps.
• for a nicer printing on REPL, a few more status_summary functions were added
• stochastic_gradient_descent now uses its retraction_method keyword.
• stochastic_gradient_descent with an absolute step length now normalizes by the stochastic gradient it steps along.
• StopWhenAll now accepts a concretely typed vector of criteria, like StopWhenAny already did, instead of silently wrapping it as a single criterion.
• StopWhenAllLanczosVectorsUsed also fires when more vectors than configured were used.
• StopWhenBestCostInGenerationConstant now resets its counter when a solver starts, so a reused criterion no longer stops in the first iteration.
• StopWhenChangeLess(ε) without a manifold now stores its iterate generically, instead of allocating storage on a DefaultManifold.
• StopWhenCriterionWithIterationCondition no longer returns the inner criterion's value from its reset call, which could stop a solver at iteration 0 without ever consulting the iteration condition.
• StopWhenIterateNaN now also checks nested point representations instead of throwing a MethodError.
• StopWhenKKTResidualLess now squares the Lagrangian gradient norm, as documented.
• StopWhenLagrangeMultiplierLess now provides the documented default tolerance 1e-6, so it can be constructed without arguments.
• StopWhenLagrangeMultiplierLess now indicates convergence, so a bundle solver that stops on it reports a converged run.
• StopWhenPopulationConcentrated and StopWhenRelativeResidualLess accept tolerances of different types and promote them.
• StopWhenPopulationCostConcentrated now compares the worst against the best fitness of the current generation.
• StopWhenPopulationDiverges now tests growth against the start of the run, not an absolute threshold.
• StopWhenRelativeAPosterioriCostChangeLessOrEqual now reports the signed relative cost change it actually tests, instead of its absolute value.
• StopWhenRelativeAPosterioriCostChangeLessOrEqual now accepts an integer factr.
• has_converged of StopWhenRepeated now requires the wrapper itself to have indicated to stop.
• recording or debugging the step size, or using StopWhenStepsizeLess, no longer re-runs the BarzilaiBorwein, CubicBracketingLinesearch, HagerZhangLinesearch and NonmonotoneLinesearch step sizes.
• subgradient_method(M, sgo) with an objective now defaults its start point to rand(M), as documented.
• subgradient_method! now leaves its result (the best visited point) in the input point, which before ended at the last iterate.
• the standalone truncated_conjugate_gradient_descent now defaults its trust region radius from the base manifold (not the flat tangent space).
• truncated_conjugate_gradient_descent now reports the trust region as exceeded after a boundary step in a negative-curvature direction, so trust_regions can enlarge its radius.
• trust_regions now includes the Hessian term of the model decrease in its acceptance ratio also in the default (non-randomized) mode and reuses the Hessian product already computed by its tCG sub solver.
• trust_regions now runs with a closed form sub solver.
• trust_regions now wraps an allocating closed-form sub solver as returning a tangent vector.
• trust_regions no longer throws for a non-tCG sub state.
• trust_regions now updates the stored gradient when a step is accepted, so gradient-based stopping criteria as well as debug and record output refer to the current iterate.
• trust_regions now builds its default sub problem from the objective after the objective_type conversion, so objective_type=:Euclidean also applies to the sub solver.
• TrustRegionModelObjective and AdaptiveRegularizationWithCubicsModelObjective now accept any AbstractManifoldHessianObjective, as documented.
• WolfePowellBinaryLinesearch now bisects correctly until the step size fulfills both Wolfe conditions; sometimes a wrong termination check made it stop too early.

Furthermore the following were fixed

• Minor bug fix in the solver adaptive_regularization_with_cubics to work on complex manifolds (#644)

0.6.6 (August 25, 2026)

Fixed

• augmented_Lagrangian_method set the penalty parameter ρ of the sub problem's cost to the constant 1/3 instead of the current alms.ρ, while its gradient did receive alms.ρ. (#637)
• fix set_parameter! for :μ and :λ for AugmentedLagrangianCost. (#637)
• fix AbstractConstrainedFunction to have two parameters, one for the constrained objective it wraps and one for the type of dual parameters (#639).

0.6.5 (August 22, 2026)

Added

• a new function requires_update that per <:StoppingCriterion type can indicate, whether a state is present that needs updating in every iteration.
• a new shortcut for the StopWhenCriterionWithIterationCondition that only evaluates every nth iteration using stopping_criterion ≞ n (\measeq <TAB> on REPL) as constructor. While in theory also % would have been possible, there was a discussion not to use more-common symbols here, cf. (#509) – and the m could be seen as “modulo”.

Fixed

• StopWhenAll and StopWhenAny evaluated their criteria with short-circuiting, so criteria later in the list were not called in every iteration. Stateful criteria like StopWhenChangeLess or StopWhenRepeated were hence not updated and the reset at initialization did not reach them either. Now requires_update is used to determine which ones need to be called even after one could use a shortcut already – only the ones without state are “short-circuited”. (#632) (#635)

0.6.4 (August 21, 2026)

As an overarching scheme of this release, the single functions in an objective become more independent; their wrapping happens automatically and the objective gets – in turn – a bit more lightweight and has “less cases to handle”.

Added

• DocumenterCodeBlocks.jl plugin added to the documentation
• DocumenterLandingPage.jl enhances the start page with a short teaser for the package now.
• Added a Developer Guide section in the documentation
• ManoptKeywordError is now exported, since it is the error thrown when the :KeywordsErrorMode parameter is set to "error" and a solver receives a keyword it does not accept.

Changed

• All AbstractManifoldObjectives had an evaluation type as first parameter. This has been refactored “down to” the function level, and the functions in the objective are now all expected to be in-place. This allows to combine allocating and in-place variants. The constructors, however, have not changed. (#630)
• the file structure has been changed and the plan/ folder has been split with the following motivation
  • all abstract types and generic implementation and documentation of functions has been moved to base/. This is also reflected in the documentation, where the new base/ files reflect a developer documentation that additionally includes descriptions of the design choices
  • concrete types and their implementations have either been moved to the specific solver where they are used / defined, especially for solver states, or now reside in a commons/ folder, when they are of general use for multiple solvers. This structure is also reflected in the documentation.
• a few internal abstract super types have been renamed for the new scheme that puts more focus on functions, to stay more consistent. The word “functor” is now avoided for structs that actually just represent functions.
  • AbstractConstrainedFunctor has been renamed to AbstractConstrainedFunction
  • AbstractConstrainedSlackFunctor has been renamed to AbstractConstrainedSlackFunction
• get_gradient_function and get_hessian_function are unified to always return an allocating variant by default, but can also return an in-place variant now based on an evaluation= keyword. While this is formally breaking, since the default behavior changed, both functions are internal and should also only be used within Manopt.jl.
• objectives now also accept a p= keyword to automatically “wrap” functions that operate on immutable variables – internally Manopt.jl expects points and tangent vectors to be mutable.

Fixed

• has_converged produced inconsistent behaviors for more complex stopping criteria. (#631) This has been fixed and the function works now consistently for both StopWhenAny and StopWhenAll. (#630)
• both in-place subsolver for the convex bundle and proximal bundle method would fail to update the result in case the size of the vector changes. this is now adapted by resizing the (in-place, passed down) result vector accordingly for these subsolvers.
• the cached and the counting objective dispatched the inequality constraint length onto equality_constraints_length. Cache sizes and counters were therefore wrong whenever a problem has different numbers of equality and inequality constraints.
• the default of start_enforcing_wolfe_conditions_at_bracketing_iteration in HagerZhangLinesearch was chosen by testing initial_guess isa ConstantStepsize, which can never hold, since ConstantInitialGuess is not a Stepsize. The default was hence always 1 instead of the intended 2 for a constant initial guess.
• the DefaultManifold fallback of the vector norm used in the Hager-Zhang initial guess computed the norm of the point instead of the norm of the tangent vector.
• several show/status_summary methods errored or printed wrong content: StopWhenEntryChangeLess and StopWhenRepeated threw on some context= values, StopWhenLagrangeMultiplierLess printed Base.names instead of its own field, the two Wolfe-Powell line searches printed the retraction method under “vector transport method”, and the objective decorators dropped the context= keyword instead of passing it on.
• a larger number of docstrings were corrected, where signatures, field names, or supertypes had drifted from the implementation, together with formula and markup errors.

Removed

• The two evaluation types of ParentEvaluationType and AllocatingInplaceEvaluation were never used anywhere, so removing them is considered nonbreaking. Internally this distinction is now anyways handled on a function level, so that neither of the cases can appear anyways. Neither of these types was ever exported.

0.6.3 (July 20, 2026)

Added

• an extra condition in the backtracking ProximalGradientMethodBacktrackingStepsize for the proximal_gradient_method. (#629)

0.6.2 (July 17, 2026)

Added

• a callbacks = keyword to all solvers (#626)

Changed

• Replaced internal ZeroTangentVector with ZeroVector from ManifoldsBase.jl. (#625)

• internally restructure the plans/ code folder and documentation and split it into two parts (#626)

  • a base/ folder defining interfaces and documenting design ideas of these
  • a commons/ folder collecting things (some or all) solvers have in common

  this should not break other peoples code, but in the documentation, namely “hard link”s to any plans/ element might break – and will all break in the future. When you are linking to these from another documentation, consider using DocumenterInterlinks instead.

Deprecated

• with actual callbacks available, the DebugCallback action from before is now obsolete and deprecated (#626)
• the callback = is deprecated, since it only served DebugCallback (#626)

0.6.1 (July 4, 2026)

Added

• the gradient sampling algorithm based on the paper by Hosseini and Uschmajew (#618)

Fixed

• Fix a typo in Riemannian Levenberg–Marquardt documentation. (#621)
• Fixed a coordinate-cache allocation in the VectorDifferentialFunction Jacobian-action path. (#622)

0.6.0 (June 24, 2026)

This is a breaking change since the JuMP extension is dropped. We also unified a few of the internal solver state constructors.

Added

• A robustified version of the Riemannian Levenberg Marquardt algorithm (#617)
• An option to disable the warm start the conjugate residual currently does when used as a subsolver.
• nonpositive_curvature_behavior for QuasiNewtonLimitedMemoryDirectionUpdate that determines how transported (y, s) vector pairs are treated after transport; if their inner product gets too low, it may lead to non-positive-definite Hessians which needs to be avoided. This resolves issue (#549). (#554)
• GeneralizedCauchyDirectionSubsolver for handling direction selection in the presence of box (Hyperrectangle) constraints in quasi-Newton methods. This allows for L-BFGS-B-style box constraint handling. (#554)
• New stopping criteria: StopWhenRelativeAPosterioriCostChangeLessOrEqual and StopWhenProjectedNegativeGradientNormLess. (#554).
• HagerZhangLinesearch stepsize, a state-of-the-art line search for smooth objectives with cubic interpolation and adaptive Wolfe condition checking. (#554)
• Step sizes can now be initialized using initialize_stepsize!, similar to solvers. (#554)
• The ConjugateResidualState now has a warm_start= option when used multiple times, for example in every iteration as a subsolver, to reuse the last state from the previous run.

Changed

• In the Riemannian Levenberg Marquardt algorithm the η parameter has been renamed to candidate_acceptance_threshold, β to damping_increase_factor and β_reduction to damping_reduction_factor. (#617)
• the constructor for the Levenberg-Marquardt state has been unified with the remaining states, to take the sub_problem and sub_state arguments as second and third positional arguments, respectively. (#617)
• the keyword initial_jacobian_f within LevenbergMarquardt is unified in naming to the residual values vector and called initial_jacobian_matrices. If you call LevenbergMarquardt with a single vector component, also a single matrix is allowed. (#617)
• an internal field of the solver state of Levenberg-Marquardt was called jacobian_f the same as the functions whose result it meant to cache if applicable. To distinguish both, the field is now called jacobian_matrices. (#617)
• the max_stepsize(M) on the SymmetricPositiveDefinite manifold was changed from returning Inf, which is the mathematical maximal stepsize to returning the square root of the maximum (floating point) value to avoid numerical instabilities.
• title of "How to define the cost in the embedding" tutorial (#615)

Fixed

• Fixed show methods of various state and stopping criteria to properly handle both repr and multiline printing (#569)
• Unified all show methods and their human readable analoga status_summary throughout the package (#569)
• Fixed some text descriptions of a few stopping criteria.
• unify naming of fields, debugDictionary of the debug state is now called debug_dictionary
• the NesterovRule now also stores an actual AbstractRetractionMethod instead of implicitly always using the default one.
• Line searches consistently respect stop_when_stepsize_exceeds keyword argument as a hard limit. (#554)
• StopWhenChangeLess falsely claimed to indicate convergence. This is now fixed. (#554)
• miscellaneous broken links in the documentation. (#614)

Removed

• The extension to JuMP. A replacement as a separate package is planned when the support for variables beyond vectors is more accessible in JuMP
• the plotting functions to Asymptote. They can now be found in the separate package ManifoldAsymptote.jl this way, Manopt.jl has less dependencies, especially the color and colorschemes dependencies are dropped
• linear_subsolver! = was removed from the LevenbergMarquardt solver interface, since it is imprecise. If you use a closed form solver before, specify it by passing the function to sub_problem and set sub_state to the corresponding evaluation type

0.5.39 (June 3, 2026)

Fixed

• a small bug where debug statements were printed even though they should not be due to DebugEvery and unified warnings to print independent of DebugEvery. (#609)

0.5.38 (May 19, 2026)

Changed

• the :convex backtracking strategy for proximal_gradient_method now entails a slightly different condition whenever the upper bound on the sectional curvature of the manifold, input via the k_max kwarg, is positive. This comes with a "tolerance" type parameter, δ, which must be positive.

0.5.37 (May 5, 2026)

Changed

• The default restart rule for conjugate_gradient_descent is now RestartOnNonDescent instead of NeverRestart, which makes the algorithm more robust to non-convexity and numerical issues. The old default can still be used by explicitly passing restart_condition=NeverRestart(). (#604)
• HagerZhangCoefficientRule now has a safeguard against the denominator being too close to zero (the denom_threshold field). By default it is set to 1.0e-10. You can set it to a lower positive value (or even zero) to weaken the safeguard, but it is recommended to keep it to avoid numerical issues. (#604)
• introduce for all Rules also a variant without being encapsulated in a memory, where the old values have to be passed as keywords. This is now used by the ConjugateGradientBealeRestartRule when evaluating its inner rule. (#604)

0.5.36 (April 24, 2026)

Added

• a function stopped_at(state) to access the number of iterations it took a solver to stop. (#599)

Fixed

• a small bug where get_count(sc::StopWhenAny, Val(:Iteration)) wrongly reported it stopped before the first iteration when it actually did not yet stop. (#599)

0.5.35 (April 16, 2026)

Changed

• NonlinearLeastSquaresObjective is now called ManifoldNonlinearLeastSquaresObjective (#569).
• (breaking) discontinue the JuMP extension. (#532)
• Improved formatting of the references in the Readme.md (#586)
• Bump compat for RecursiveArrayTools.jl to include version 4
• deactivate CompatHelper Action and solely use dependabot
• (breaking change) renamed CoordinateVectorialType to CoefficientVectorialType to have a consistent naming that anything with respect to a basis is called “coefficients”
• moved the old closed-form-in-coordinates subsolver for LevenbergMarquardt handling to the subsolver; if you implemented your own, pass it to sub_problem,

set the sub_state to indicate allocating or in-place evaluation and change the signature as documented; make especially sure to return a tangent vector now and not coordinates.

• (breaking change) expect_zero_residual in LevenbergMarquardt is replaced by more general damping_reduction_threshold and β_reduction. To recover the behavior of expect_zero_residual=true, set damping_reduction_threshold to the same value as η and β_reduction to β.

Fixed

• The default line search in conjugate_gradient_descent is now ArmijoLinesearchStepsize instead of ArmijoLinesearch, which makes it work well with custom point types.

0.5.34 (March 3, 2026)

Fixed

• Float32 support in trust_regions solver was broken in the previous release, which is now fixed.

0.5.33 (February 18, 2026)

Added

• A clarification on the use of AI in the CONTRIBUTING.md (#573)
• _produce_type now accepts the point p as an optional third argument, which can be used to produce objects with specific point type for internal buffers. The addition has been utilized in DirectionUpdateRules and Stepsizes to improve GPU and custom floating point type compatibility. (#577)
• Added another package and paper using Manopt.jl to the about page (#576).

Fixed

• DistanceOverGradientsStepsize now requires explicitly passing a point as the second argument because it logically depends on receiving the initial point. (#577)

0.5.32 (January 15, 2026)

Fixed

• Fixed failing precompilation related to the release of Glossaries.jl v0.1.1 (#567).

0.5.31 (January 11, 2026)

Changed

Moved the documentation glossaries to using the new Glossaries.jl package.

0.5.30 (December 10, 2025)

Added

• add keyword argument is_feasible_error to interior_point_Newton to control how to handle infeasible starting points (#556)
• add keyword argument at_init to some debug options to control whether they print already at the initialisation and hence before the first iteration (#552)

Fixed

• fixed a few typos in the documentation (#557)
• fixed a bug in StopWhenRepeated where it stopped already at initialisation if the interior stopping criterion was satisfied (#558)

0.5.29 (November 26, 2025)

Added

• a keyword argument atol to the ConstrainedManifoldObjective to set a tolerance for constraint satisfaction. (#545)
• a spell checker following crate-ci/typos

Fixed

• Fixed a typo in DebugFeasibility, where an undefined variable was used. (#544)

Changed

• Removed atol from DebugFeasibility and instead use the one newly added atol from the ConstrainedManifoldObjective. (#546)
• Move from CompatHelper to dependabot to keep track of dependency updates in Julia packages. (#547)
• moved the ManoptTestSuite module to a sub module Manopt.Test within Manopt.jl, so it can be easier reused by others as well (#550)
• moved to using a Project.toml for tests and an overall [Workspace]. This also allows finally to run single test files without installing all packages manually, but instead just switching to and instantiating the test environment. (#550)
• for compatibility, state also [source] entries consistently in the sub Project.toml files. (#550)

0.5.28 (November 17, 2025)

Changed

• Unified the interfaces for line search related functions, especially,
  • linesearch_backtrack(M, F, p, X, s, decrease, contract, η, f0; kwargs...) now has lf0= and gradient= keyword arguments instead of positional ones for X and the last f0, respectively. It additionally has a Dlf0= keyword argument to pass the evaluated differential instead of the gradient, which otherwise defaults to calling the inner product.
• Refactor the nonmonotone linesearch stepsize to have an initial guess that can be set. For now it still afterwards performs the Barzilai-Borwein initial guess,

so a constant initial guess is recommended here. The initial guess may be refactored in the future in a non-breaking release and the meaning of the initial guess in nonmonotone line search may change.

Fixed

• Change the construction of the product manifold in interior_point_Newton from × to ProductManifold, so that the algorithm also work on Product manifolds M, where it now correctly wraps M instead of extending it.
• Unified the doc strings for constrained problems.
• Fixed a few typos in the doc strings of matrix update formulae within the quasi-Newton and CG solver.
• Covered one last line in proximal_gradient_plan

0.5.27 (November 11, 2025)

Added

• In WolfePowellLinesearchStepsize, two new keyword arguments stop_increasing_at_step= and stop_decreasing_at_step= were added to limit the number of increase/decrease steps in the initial bracketing phase for splus and sminus, respectively. (resolves (#495))
• refactor get_message to only allocate a string when it is asked to deliver one, not every time a message is actually stored. This makes the message system align more with get_reason.

0.5.26 (November 5, 2025)

Added

• a vectorbundle_newton solver to find zeros of equations defined on vector bundles.

Fixed

• fixes a few inconsistencies regarding get_embedding, which now consistently uses a point type as positional second argument.

Changed

• fixed a few typos in the documentation strings of a few solvers.
• fixed a typo in the documentation of LevenbergMarquardt.
• fixed a bug in an internal tex command to print sums in the documentation.
• fixed the use of mesh_adaptive_direct_search on manifolds with irrational injectivity radius.
• improved the CONTRIBUTING.md to reflect the new code formatter we use, as mentioned in (#527).

0.5.25 (October 9, 2025)

Changed

• Bumped dependencies of all JuliaManifolds ecosystem packages to be consistent with ManifoldsBase.jl 2.0 and Manifolds.jl 0.11

0.5.24 (October 6, 2025)

Added

• CubicBracketingLinesearch step size
• fallback in proximal_gradient_plan to use the norm of the inverse retraction if the distance is not available.

0.5.23 (September 14, 2025)

Added

• HybridCoefficient(args...) conjugate gradient parameters.
• a function has_converged(sc) function for any StoppingCriterion to indicate that it both has stopped and the reason is a convergence certificate. Note that compared to the static evaluation of indicates_convergence(sc), which is independent of the state of the criterion, this is the dynamic variant to be used after a solver has stopped.
• a has_converged(::AbstractManoptSolverState) function to check whether the solver has converged.

Changed

• formerly a stopping criterion could be activated at certain iterations with sc > 5, sc >= 5, sc == 5, sc <= 5, and sc < 5. This caused too many issues with invalidations, so it has been reduced and moved to sc ⩼ 5, sc ≟ 5, sc ⩻ 5 for the cases 1, 3, and 5, respectively, cf. (#509).
• Refine the JuMP extension and add an allocation-free cost and gradient callback for JuMP interface (#498)

0.5.22 (September 09, 2025)

Added

• a keywords_accepted(f, mode=:warn; kwargs...) function that verifies that all keywords are accepted by a certain function.
• an internal function calls_with_kwargs(f) to indicate which functions f passes kwargs... to.
• a KeywordsErrorMode preference parameter to control how keywords that are not used/allowed should be treated. Values are "none", "warn" (default), and "error".
• Add Distance over Gradients (RDoG) stepsize: DistanceOverGradientsStepsize and factory DistanceOverGradients, a learning‑rate‑free, curvature‑aware stepsize with show/repr and tests on Euclidean, Sphere, and Hyperbolic manifolds.

Fixed

• the typo in the name AdaptiveRgularizationWithCubicsModelObjective is fixed to AdaptiveRegularizationWithCubicsModelObjective.

0.5.21 (September 5, 2025)

Added

• a system to track keywords, warning when unused ones are passed and a static way to explore possible keywords.
• a warm_start_factor field to ProximalGradientMethodBacktrackingStepsize to allow to scale the stepsize in the backtracking procedure.
• a gradient= keyword in several Stepsizes, such that one can avoid to internally avoid computing the gradient again.
• used the gradient= keyword in
  • alternating_gradient_descent
  • conjugate_gradient_descent
  • Frank_Wolfe_method
  • gradient_descent
  • interior_point_Newton
  • quasi_Newton
  • projected_gradient_method
• a restart_condition functor to conjugate_gradient_descent, which allows the algorithm to restart if the search direction is sub-par (#492)
• two literature references

Changed

• remodeled the docs for the extensions a bit, added JuMP to the DocumenterInterlinks.
• the internal VectorizedManifold within that extension is now called ManifoldSet
• the internal ArrayShape within that extension is now called ManifoldPointArrayShape
• Switch to using Runic.jl as code formatter

Fixed

• Fixed some math rendering in the docs, especially avoid raw strings and interpolate math symbols more often.
• Fixed allocations in the callbacks of the JuMP interface so that the solver can query the cost and gradient without allocating.

0.5.20 (July 8, 2025)

Added

• a DebugWarnIfStepsizeCollapsed DebugAction and a related :WarnStepsize symbol for the debug dictionary. This is to be used in conjunction with the ProximalGradientMethodBacktracking stepsize to warn if the backtracking procedure of the proximal_gradient_method hit the stepsize length threshold without converging.

Changed

• bumped dependencies.

Fixed

• Fixed a few typos in the docs.

0.5.19 (July 4, 2025)

Added

• a function get_differential and get_differential_function for first order objectives.
• a ParentEvaluationType to indicate that a certain objective inherits it evaluation from the parent (wrapping) objective
• a new AllocatingInplaceEvaluation that is used for the functions that offer both variants simultaneously.
• a differential= keyword for providing a faster way of computing inner(M, p, grad_f(p), X), introduced to the algorithms conjugate_gradient_descent, gradient_descent, Frank_Wolfe_method, quasi_Newton

Changed

• the ManifoldGradientObjective and the ManifoldCostGradientObjective are now merely a const special cases of the ManifoldFirstOrderObjective, since this type might now also represent a differential or other combinations of cost, grad, and differential, where they are computed together.
• the AbstractManifoldGradientObjective is renamed to AbstractManifoldFirstOrderObjective, since the

second function might now also represent a differential.

Fixed

• fixes a small bug where calling mesh_adaptive_direct_search with a start point in some cases did not initialize the state correctly with that start point.
• The HestenesStiefelCoefficient now also always returns a real value, similar the other coefficient rules. To the best of our knowledge, this might have been a bug previously.

0.5.18 (June 18, 2025)

Added

• Introduce the algorithm proximal_gradient_method along with ManifoldProximalGradientObjective, ProximalGradientMethodState, as well as an experimental ProximalGradientMethodAcceleration.
• Add ProximalGradientMethodBacktracking stepsize.
• Add StopWhenGradientMappingNormLess stopping criterion.
• Introduce a StopWhenRepeated stopping criterion that stops when the given stopping criterion has indicated to stop n times (consecutively, if consecutive=true).
• Introduce a StopWhenCriterionWithIterationCondition stopping criterion that stops when a given stopping criterion has been satisfied together with a certain iteration condition. This can then be generated even with shortcuts like sc > 5
• Introduce a DebugCallback that allows to add a callback function to the debug system
• Introduce a callback= keyword to all solvers.
• Added back functions estimate_sectional_curvature, ζ_1, ζ_2, close_point from convex_bundle_method; the function call can stay the same as before since there is a curvature estimation fallback
• Add back some fields and arguments such as p_estimate, ϱ, α, from ConvexBundleMethodState

Changed

• make the GradientDescentState a bit more tolerant to ignore keywords it does not use.

0.5.17 (June 3, 2025)

Added

• Introduce a StopWhenCostChangeLess stopping criterion that stops when the cost function changes less than a given value.

0.5.16 (May 7, 2025)

Fixed

• fixes a bug in the LineSearches.jl extension, where two (old) retract!s were still

present; they were changed to retract_fused!.

0.5.15 (May 6, 2025)

Fixed

• CMA-ES no longer errors when the covariance matrix has nonpositive eigenvalues due to numerical issues.

0.5.14 (May 5, 2025)

Added

• linear_subsolver! is added as a keyword argument to the Levenberg-Marquardt interface.

Changed

• adapt to using default_basis where appropriate.
• the tutorials are now rendered with quarto using the QuartoNotebookRunner.jl and are hence purely julia based.

0.5.13 (April 25, 2025)

Added

• Allow setting AbstractManifoldObjective through JuMP

Changed

• Remove dependency on ManoptExamples.jl which yielded a circular dependency, though only through extras
• Unify dummy types and several test functions into the ManoptTestSuite subpackage.

Fixed

• A scaling error that appeared only when calling get_cost_function on the new ScaledManifoldObjective.
• Documentation issues for quasi-Newton solvers.
• fixes a scaling error in quasi newton
• Fixes printing of JuMP models containing Manopt solver.

0.5.12 (April 13, 2025)

Added

• a ScaledManifoldObjective to easier build scaled versions of objectives, especially turn maximisation problems into minimisation ones using a scaling of -1.
• Introduce a ManifoldConstrainedSetObjective
• Introduce a projected_gradient_method

0.5.11 (April 8, 2025)

Added

• Configurable subsolver for the linear subproblem in Levenberg-Marquardt. The default subsolver is now also robust to numerical issues that may cause Cholesky decomposition to fail.

0.5.10 (April 4, 2025)

Fixed

• a proper implementation of the preconditioning for quasi_Newton, that can be used instead of or in combination with the initial scaling.

0.5.9 (March 24, 2025)

Added

• add a PreconditionedDirection variant to the direction gradient processor keyword argument and its corresponding PreconditionedDirectionRule
• make the preconditioner available in quasi Newton.
• in gradient_descent and conjugate_gradient_descent the rule can be added anyway.

Fixed

• the links in the AD tutorial are fixed and moved to using extref

0.5.8 (February 28, 2025)

Fixed

• fixed a small bug in the NonmonotoneLinesearchStepsize when the injectivity radius is an irrational number.
• fixed a small bug in check_gradient where eps might have been called on complex types.
• fixed a bug in several gradient based solvers like quasi_newton, such that they properly work with the combined cost grad objective.
• fixes a few typos in the docs.

0.5.7 (February 20, 2025)

Added

• Adds a mesh adaptive direct search algorithm (MADS), using the LTMADS variant with a lower triangular (LT) random matrix in the mesh generation.

0.5.6 (February 10, 2025)

Changed

• bump dependencies of all JuliaManifolds ecosystem packages to be consistent with ManifoldsBase 1.0

0.5.5 (January 4, 2025)

Added

• the Levenberg-Marquardt algorithm internally uses a VectorGradientFunction, which allows

to use a vector of gradients of a function returning all gradients as well for the algorithm

• The VectorGradientFunction now also have a get_jacobian function

Changed

• Minimum Julia version is now 1.10 (the LTS which replaced 1.6)
• The vectorial functions had a bug where the original vector function for the mutating case was not always treated as mutating.

Removed

• The geodesic regression example, first because it is not correct, second because it should become part of ManoptExamples.jl once it is correct.

0.5.4 (December 11, 2024)

Added

• An automated detection whether the tutorials are present if not an also no quarto run is done, an automated --exclude-tutorials option is added.
• Support for ManifoldDiff 0.4
• icons upfront external links when they link to another package or Wikipedia.

0.5.3 (October 18, 2024)

Added

• StopWhenChangeLess, StopWhenGradientChangeLess and StopWhenGradientLess can now use the new idea (ManifoldsBase.jl 0.15.18) of different outer norms on manifolds with components like power and product manifolds and all others that support this from the Manifolds.jl Library, like Euclidean

Changed

• stabilize max_stepsize to also work when injectivity_radius does not exist. It however would warn new users, that activate tutorial mode.
• Start a ManoptTestSuite sub package to store dummy types and common test helpers in.

0.5.2 (October 5, 2024)

Added

• three new symbols to easier state to record the :Gradient, the :GradientNorm, and the :Stepsize.

Changed

• fix a few typos in the documentation
• improved the documentation for the initial guess of ArmijoLinesearchStepsize.

0.5.1 (September 4, 2024)

Changed

• slightly improves the test for the ExponentialFamilyProjection text on the about page.

Added

• the proximal_point method.

0.5.0 (August 29, 2024)

This breaking update is mainly concerned with improving a unified experience through all solvers and some usability improvements, such that for example the different gradient update rules are easier to specify.

In general this introduces a few factories, that avoid having to pass the manifold to keyword arguments

Added

• A ManifoldDefaultsFactory that postpones the creation/allocation of manifold-specific fields in for example direction updates, step sizes and stopping criteria. As a rule of thumb, internal structures, like a solver state should store the final type. Any high-level interface, like the functions to start solvers, should accept such a factory in the appropriate places and call the internal _produce_type(factory, M), for example before passing something to the state.
• a documentation_glossary.jl file containing a glossary of often used variables in fields, arguments, and keywords, to print them in a unified manner. The same for usual sections, text, and math notation that is often used within the doc-strings.

Changed

• Any Stepsize now has a Stepsize struct used internally as the original structs before. The newly exported terms aim to fit stepsize=... in naming and create a ManifoldDefaultsFactory instead, so that any stepsize can be created without explicitly specifying the manifold.
  • ConstantStepsize is no longer exported, use ConstantLength instead. The length parameter is now a positional argument following the (optional) manifold. Besides that ConstantLength works as before, just that omitting the manifold fills the one specified in the solver now.
  • DecreasingStepsize is no longer exported, use DecreasingLength instead. DecreasingLength works as before, just that omitting the manifold fills the one specified in the solver now.
  • ArmijoLinesearch is now called ArmijoLinesearchStepsize. ArmijoLinesearch works as before, just that omitting the manifold fills the one specified in the solver now.
  • WolfePowellLinesearch is now called WolfePowellLinesearchStepsize, its constant c_1 is now unified with Armijo and called sufficient_decrease, c_2 was renamed to sufficient_curvature. Besides that, WolfePowellLinesearch works as before, just that omitting the manifold fills the one specified in the solver now.
  • WolfePowellBinaryLinesearch is now called WolfePowellBinaryLinesearchStepsize, its constant c_1 is now unified with Armijo and called sufficient_decrease, c_2 was renamed to sufficient_curvature. Besides that, WolfePowellBinaryLinesearch works as before, just that omitting the manifold fills the one specified in the solver now.
  • NonmonotoneLinesearch is now called NonmonotoneLinesearchStepsize. NonmonotoneLinesearch works as before, just that omitting the manifold fills the one specified in the solver now.
  • AdaptiveWNGradient is now called AdaptiveWNGradientStepsize. Its second positional argument, the gradient function was only evaluated once for the gradient_bound default, so it has been replaced by the keyword X= accepting a tangent vector. The last positional argument p has also been moved to a keyword argument. Besides that, AdaptiveWNGradient works as before, just that omitting the manifold fills the one specified in the solver now.
• Any DirectionUpdateRule now has the Rule in its name, since the original name is used to create the ManifoldDefaultsFactory instead. The original constructor now no longer requires the manifold as a parameter, that is later done in the factory. The Rule is, however, also no longer exported.
  • AverageGradient is now called AverageGradientRule. AverageGradient works as before, but the manifold as its first parameter is no longer necessary and p is now a keyword argument.
  • The IdentityUpdateRule now accepts a manifold optionally for consistency, and you can use Gradient() for short as well as its factory. Hence direction=Gradient() is now available.
  • MomentumGradient is now called MomentumGradientRule. MomentumGradient works as before, but the manifold as its first parameter is no longer necessary and p is now a keyword argument.
  • Nesterov is now called NesterovRule. Nesterov works as before, but the manifold as its first parameter is no longer necessary and p is now a keyword argument.
  • ConjugateDescentCoefficient is now called ConjugateDescentCoefficientRule. ConjugateDescentCoefficient works as before, but can now use the factory in between
  • the ConjugateGradientBealeRestart is now called ConjugateGradientBealeRestartRule. For the ConjugateGradientBealeRestart the manifold is now a first parameter, that is not necessary and no longer the manifold= keyword.
  • DaiYuanCoefficient is now called DaiYuanCoefficientRule. For the DaiYuanCoefficient the manifold as its first parameter is no longer necessary and the vector transport has been unified/moved to the vector_transport_method= keyword.
  • FletcherReevesCoefficient is now called FletcherReevesCoefficientRule. FletcherReevesCoefficient works as before, but can now use the factory in between
  • HagerZhangCoefficient is now called HagerZhangCoefficientRule. For the HagerZhangCoefficient the manifold as its first parameter is no longer necessary and the vector transport has been unified/moved to the vector_transport_method= keyword.
  • HestenesStiefelCoefficient is now called HestenesStiefelCoefficientRule. For the HestenesStiefelCoefficient the manifold as its first parameter is no longer necessary and the vector transport has been unified/moved to the vector_transport_method= keyword.
  • LiuStoreyCoefficient is now called LiuStoreyCoefficientRule. For the LiuStoreyCoefficient the manifold as its first parameter is no longer necessary and the vector transport has been unified/moved to the vector_transport_method= keyword.
  • PolakRibiereCoefficient is now called PolakRibiereCoefficientRule. For the PolakRibiereCoefficient the manifold as its first parameter is no longer necessary and the vector transport has been unified/moved to the vector_transport_method= keyword.
  • the SteepestDirectionUpdateRule is now called SteepestDescentCoefficientRule. The SteepestDescentCoefficient is equivalent, but creates the new factory temporarily.
  • AbstractGradientGroupProcessor is now called AbstractGradientGroupDirectionRule
    • the StochasticGradient is now called StochasticGradientRule. The StochasticGradient is equivalent, but creates the new factory temporarily, so that the manifold is not longer necessary.
  • the AlternatingGradient is now called AlternatingGradientRule.
  The AlternatingGradient is equivalent, but creates the new factory temporarily, so that the manifold is not longer necessary.
• quasi_Newton had a keyword scale_initial_operator= that was inconsistently declared (sometimes boolean, sometimes real) and was unused. It is now called initial_scale=1.0 and scales the initial (diagonal, unit) matrix within the approximation of the Hessian additionally to the $\frac{1}{\lVert g_k\rVert}$ scaling with the norm of the oldest gradient for the limited memory variant. For the full matrix variant the initial identity matrix is now scaled with this parameter.
• Unify doc strings and presentation of keyword arguments
  • general indexing, for example in a vector, uses i
  • index for inequality constraints is unified to i running from 1,...,m
  • index for equality constraints is unified to j running from 1,...,n
  • iterations are using now k
• get_manopt_parameter has been renamed to get_parameter since it is internal, so internally that is clear; accessing it from outside hence reads anyway Manopt.get_parameter
• set_manopt_parameter! has been renamed to set_parameter! since it is internal, so internally that is clear; accessing it from outside hence reads Manopt.set_parameter!
• changed the stabilize::Bool= keyword in quasi_Newton to the more flexible project!= keyword, this is also more in line with the other solvers. Internally the same is done within the QuasiNewtonLimitedMemoryDirectionUpdate. To adapt,
  • the previous stabilize=true is now set with (project!)=embed_project! in general, and if the manifold is represented by points in the embedding, like the sphere, (project!)=project! suffices
  • the new default is (project!)=copyto!, so by default no projection/stabilization is performed.
• the positional argument p (usually the last or the third to last if sub solvers existed) has been moved to a keyword argument p= in all State constructors
• in NelderMeadState the population moved from positional to keyword argument as well,
• the way to initialize sub solvers in the solver states has been unified In the new variant
  • the sub_problem is always a positional argument; namely the last one
  • if the sub_state is given as a optional positional argument after the problem, it has to be a manopt solver state
  • you can provide the new ClosedFormSubSolverState(e::AbstractEvaluationType) for the state to indicate that the sub_problem is a closed form solution (function call) and how it has to be called
  • if you do not provide the sub_state as positional, the keyword evaluation= is used to generate the state ClosedFormSubSolverState.
  • when previously p and eventually X where positional arguments, they are now moved to keyword arguments of the same name for start point and tangent vector.
  • in detail
    • AdaptiveRegularizationState(M, sub_problem [, sub_state]; kwargs...) replaces the (unused) variant to only provide the objective; both X and p moved to keyword arguments.
    • AugmentedLagrangianMethodState(M, objective, sub_problem; evaluation=...) was added
    • AugmentedLagrangianMethodState(M, objective, sub_problem, sub_state; evaluation=...) now has p=rand(M) as keyword argument instead of being the second positional one
    • ExactPenaltyMethodState(M, sub_problem; evaluation=...) was added and ExactPenaltyMethodState(M, sub_problem, sub_state; evaluation=...) now has p=rand(M) as keyword argument instead of being the second positional one
    • DifferenceOfConvexState(M, sub_problem; evaluation=...) was added and DifferenceOfConvexState(M, sub_problem, sub_state; evaluation=...) now has p=rand(M) as keyword argument instead of being the second positional one
    • DifferenceOfConvexProximalState(M, sub_problem; evaluation=...) was added and DifferenceOfConvexProximalState(M, sub_problem, sub_state; evaluation=...) now has p=rand(M) as keyword argument instead of being the second positional one
  • bumped Manifolds.jlto version 0.10; this mainly means that any algorithm working on a product manifold and requiring ArrayPartition now has to explicitly do using RecursiveArrayTools.

Fixed

• the AverageGradientRule filled its internal vector of gradients wrongly or mixed it up in parallel transport. This is now fixed.

Removed

• the convex_bundle_method and its ConvexBundleMethodState no longer accept the keywords k_size, p_estimate nor ϱ, they are superseded by just providing k_max.
• the truncated_conjugate_gradient_descent(M, f, grad_f, hess_f) has the Hessian now a mandatory argument. To use the old variant, provide ApproxHessianFiniteDifference(M, copy(M, p), grad_f) to hess_f directly.
• all deprecated keyword arguments and a few function signatures were removed:
  • get_equality_constraints, get_equality_constraints!, get_inequality_constraints, get_inequality_constraints! are removed. Use their singular forms and set the index to : instead.
  • StopWhenChangeLess(ε) is removed, use StopWhenChangeLess(M, ε) instead to fill for example the retraction properly used to determine the change
• In the WolfePowellLinesearch and WolfeBinaryLinesearchthe linesearch_stopsize= keyword is replaced by stop_when_stepsize_less=
• DebugChange and RecordChange had a manifold= and a invretr keyword that were replaced by the first positional argument M and inverse_retraction_method=, respectively
• in the NonlinearLeastSquaresObjective and LevenbergMarquardt the jacB= keyword is now called jacobian_tangent_basis=
• in particle_swarm the n= keyword is replaced by swarm_size=.
• update_stopping_criterion! has been removed and unified with set_parameter!. The code adaptions are
  • to set a parameter of a stopping criterion, just replace update_stopping_criterion!(sc, :Val, v) with set_parameter!(sc, :Val, v)
  • to update a stopping criterion in a solver state, replace the old update_stopping_criterion!(state, :Val, v) that passed down to the stopping criterion by the explicit pass down with set_parameter!(state, :StoppingCriterion, :Val, v)

0.4.69 (August 3, 2024)

Changed

• Improved performance of Interior Point Newton Method.

0.4.68 (August 2, 2024)

Added

• an Interior Point Newton Method, the interior_point_Newton
• a conjugate_residual Algorithm to solve a linear system on a tangent space.
• ArmijoLinesearch now allows for additional additional_decrease_condition and additional_increase_condition keywords to add further conditions to accept additional conditions when to accept an decreasing or increase of the stepsize.
• add a DebugFeasibility to have a debug print about feasibility of points in constrained optimization employing the new is_feasible function
• add a InteriorPointCentralityCondition that can be added for step candidates within the line search of interior_point_Newton
• Add Several new functors
  • the LagrangianCost, LagrangianGradient, LagrangianHessian, that based on a constrained objective allow to construct the Hessian objective of its Lagrangian
  • the CondensedKKTVectorField and its CondensedKKTVectorFieldJacobian, that are being used to solve a linear system within interior_point_Newton
  • the KKTVectorField as well as its KKTVectorFieldJacobian and KKTVectorFieldAdjointJacobian
  • the KKTVectorFieldNormSq and its KKTVectorFieldNormSqGradient used within the Armijo line search of interior_point_Newton
• New stopping criteria
  • A StopWhenRelativeResidualLess for the conjugate_residual
  • A StopWhenKKTResidualLess for the interior_point_Newton

0.4.67 (July 25, 2024)

Added

• max_stepsize methods for Hyperrectangle.

Fixed

• a few typos in the documentation
• WolfePowellLinesearch no longer uses max_stepsize with invalid point by default.

0.4.66 (June 27, 2024)

Changed

• Remove functions estimate_sectional_curvature, ζ_1, ζ_2, close_point from convex_bundle_method
• Remove some unused fields and arguments such as p_estimate, ϱ, α, from ConvexBundleMethodState in favor of just k_max
• Change parameter R placement in ProximalBundleMethodState to fifth position

0.4.65 (June 13, 2024)

Changed

• refactor stopping criteria to not store a sc.reason internally, but instead only generate the reason (and hence allocate a string) when actually asked for a reason.

0.4.64 (June 4, 2024)

Added

• Remodel the constraints and their gradients into separate VectorGradientFunctions to reduce code duplication and encapsulate the inner model of these functions and their gradients
• Introduce a ConstrainedManoptProblem to model different ranges for the gradients in the new VectorGradientFunctions beyond the default NestedPowerRepresentation
• introduce a VectorHessianFunction to also model that one can provide the vector of Hessians to constraints
• introduce a more flexible indexing beyond single indexing, to also include arbitrary ranges when accessing vector functions and their gradients and hence also for constraints and their gradients.

Changed

• Remodel ConstrainedManifoldObjective to store an AbstractManifoldObjective internally instead of directly f and grad_f, allowing also Hessian objectives therein and implementing access to this Hessian
• Fixed a bug that Lanczos produced NaNs when started exactly in a minimizer, since the algorithm initially divides by the gradient norm.

Deprecated

• deprecate get_grad_equality_constraints(M, o, p), use get_grad_equality_constraint(M, o, p, :) from the more flexible indexing instead.

0.4.63 (May 11, 2024)

Added

• :reinitialize_direction_update option for quasi-Newton behavior when the direction is not a descent one. It is now the new default for QuasiNewtonState.
• Quasi-Newton direction update rules are now initialized upon start of the solver with the new internal function initialize_update!.

Fixed

• ALM and EPM no longer keep a part of the quasi-Newton subsolver state between runs.

Changed

• Quasi-Newton solvers: :reinitialize_direction_update is the new default behavior in case of detection of non-descent direction instead of :step_towards_negative_gradient. :step_towards_negative_gradient is still available when explicitly set using the nondescent_direction_behavior keyword argument.

0.4.62 (May 3, 2024)

Changed

• bumped dependency of ManifoldsBase.jl to 0.15.9 and imported their numerical verify functions. This changes the throw_error keyword used internally to a error= with a symbol.

0.4.61 (April 27, 2024)

Added

• Tests use Aqua.jl to spot problems in the code
• introduce a feature-based list of solvers and reduce the details in the alphabetical list
• adds a PolyakStepsize
• added a get_subgradient for AbstractManifoldGradientObjectives since their gradient is a special case of a subgradient.

Fixed

• get_last_stepsize was defined in quite different ways that caused ambiguities. That is now internally a bit restructured and should work nicer. Internally this means that the interim dispatch on get_last_stepsize(problem, state, step, vars...) was removed. Now the only two left are get_last_stepsize(p, s, vars...) and the one directly checking get_last_stepsize(::Stepsize) for stored values.
• the accidentally exported set_manopt_parameter! is no longer exported

Changed

• get_manopt_parameter and set_manopt_parameter! have been revised and better documented, they now use more semantic symbols (with capital letters) instead of direct field access (lower letter symbols). Since these are not exported, this is considered an internal, hence non-breaking change.
  • semantic symbols are now all nouns in upper case letters
  • :active is changed to :Activity

0.4.60 (April 10, 2024)

Added

• RecordWhenActive to allow records to be deactivated during runtime, symbol :WhenActive
• RecordSubsolver to record the result of a subsolver recording in the main solver, symbol :Subsolver
• RecordStoppingReason to record the reason a solver stopped
• made the RecordFactory more flexible and quite similar to DebugFactory, such that it is now also easy to specify recordings at the end of solver runs. This can especially be used to record final states of sub solvers.

Changed

• being a bit more strict with internal tools and made the factories for record non-exported, so this is the same as for debug.

Fixed

• The name :Subsolver to generate DebugWhenActive was misleading, it is now called :WhenActive referring to “print debug only when set active, that is by the parent (main) solver”.
• the old version of specifying Symbol => RecordAction for later access was ambiguous, since

it could also mean to store the action in the dictionary under that symbol. Hence the order for access was switched to RecordAction => Symbol to resolve that ambiguity.

0.4.59 (April 7, 2024)

Added

• A Riemannian variant of the CMA-ES (Covariance Matrix Adaptation Evolutionary Strategy) algorithm, cma_es.

Fixed

• The constructor dispatch for StopWhenAny with Vector had incorrect element type assertion which was fixed.

0.4.58 (March 18, 2024)

Added

• more advanced methods to add debug to the beginning of an algorithm, a step, or the end of the algorithm with DebugAction entries at :Start, :BeforeIteration, :Iteration, and :Stop, respectively.
• Introduce a Pair-based format to add elements to these hooks, while all others are now added to :Iteration (no longer to :All)
• (planned) add an easy possibility to also record the initial stage and not only after the first iteration.

Changed

• Changed the symbol for the :Step dictionary to be :Iteration, to unify this with the symbols used in recording, and removed the :All symbol. On the fine granular scale, all but :Start debugs are now reset on init. Since these are merely internal entries in the debug dictionary, this is considered non-breaking.
• introduce a StopWhenSwarmVelocityLess stopping criterion for particle_swarm replacing the current default of the swarm change, since this is a bit more effective to compute

Fixed

• fixed the outdated documentation of TruncatedConjugateGradientState, that now correctly state that p is no longer stored, but the algorithm runs on TpM.
• implemented the missing get_iterate for TruncatedConjugateGradientState.

0.4.57 (March 15, 2024)

Changed

• convex_bundle_method uses the sectional_curvature from ManifoldsBase.jl.
• convex_bundle_method no longer has the unused k_min keyword argument.
• ManifoldsBase.jl now is running on Documenter 1.3, Manopt.jl documentation now uses DocumenterInterLinks to refer to sections and functions from ManifoldsBase.jl

Fixed

• fixes a typo that when passing sub_kwargs to trust_regions caused an error in the decoration of the sub objective.

0.4.56 (March 4, 2024)

Added

• The option :step_towards_negative_gradient for nondescent_direction_behavior in quasi-Newton solvers does no longer emit a warning by default. This has been moved to a message, that can be accessed/displayed with DebugMessages
• DebugMessages now has a second positional argument, specifying whether all messages, or just the first (:Once) should be displayed.

0.4.55 (March 3, 2024)

Added

• Option nondescent_direction_behavior for quasi-Newton solvers. By default it checks for non-descent direction which may not be handled well by some stepsize selection algorithms.

Fixed

• unified documentation, especially function signatures further.
• fixed a few typos related to math formulae in the doc strings.

0.4.54 (February 28, 2024)

Added

• convex_bundle_method optimization algorithm for non-smooth geodesically convex functions
• proximal_bundle_method optimization algorithm for non-smooth functions.
• StopWhenSubgradientNormLess, StopWhenLagrangeMultiplierLess, and stopping criteria.

Fixed

• Doc strings now follow a vale.sh policy. Though this is not fully working, this PR improves a lot of the doc strings concerning wording and spelling.

0.4.53 (February 13, 2024)

Fixed

• fixes two storage action defaults, that accidentally still tried to initialize a :Population (as modified back to :Iterate 0.4.49).
• fix a few typos in the documentation and add a reference for the subgradient method.

0.4.52 (February 5, 2024)

Added

• introduce an environment persistent way of setting global values with the set_manopt_parameter! function using Preferences.jl.
• introduce such a value named :Mode to enable a "Tutorial" mode that shall often provide more warnings and information for people getting started with optimization on manifolds

0.4.51 (January 30, 2024)

Added

• A StopWhenSubgradientNormLess stopping criterion for subgradient-based optimization.
• Allow the message= of the DebugIfEntry debug action to contain a format element to print the field in the message as well.

0.4.50 (January 26, 2024)

Fixed

• Fix Quasi Newton on complex manifolds.

0.4.49 (January 18, 2024)

Added

• A StopWhenEntryChangeLess to be able to stop on arbitrary small changes of specific fields
• generalises StopWhenGradientNormLess to accept arbitrary norm= functions
• refactor the default in particle_swarm to no longer “misuse” the iteration change, but actually the new one the :swarm entry

0.4.48 (January 16, 2024)

Fixed

• fixes an imprecision in the interface of get_iterate that sometimes led to the swarm of particle_swarm being returned as the iterate.
• refactor particle_swarm in naming and access functions to avoid this also in the future. To access the whole swarm, one now should use get_manopt_parameter(pss, :Population)

0.4.47 (January 6, 2024)

Fixed

• fixed a bug, where the retraction set in check_Hessian was not passed on to the optional inner check_gradient call, which could lead to unwanted side effects, see (#342).

0.4.46 (January 1, 2024)

Changed

• An error is thrown when a line search from LineSearches.jl reports search failure.
• Changed default stopping criterion in ALM algorithm to mitigate an issue occurring when step size is very small.
• Default memory length in default ALM subsolver is now capped at manifold dimension.
• Replaced CI testing on Julia 1.8 with testing on Julia 1.10.

Fixed

• A bug in LineSearches.jl extension leading to slower convergence.
• Fixed a bug in L-BFGS related to memory storage, which caused significantly slower convergence.

0.4.45 (December 28, 2023)

Added

• Introduce sub_kwargs and sub_stopping_criterion for trust_regions as noticed in (#336)

Changed

• WolfePowellLineSearch, ArmijoLineSearch step sizes now allocate less
• linesearch_backtrack! is now available
• Quasi Newton Updates can work in-place of a direction vector as well.
• Faster safe_indices in L-BFGS.

0.4.44 (December 12, 2023)

Formally one could consider this version breaking, since a few functions have been moved, that in earlier versions (0.3.x) have been used in example scripts. These examples are now available again within ManoptExamples.jl, and with their “reappearance” the corresponding costs, gradients, differentials, adjoint differentials, and proximal maps have been moved there as well. This is not considered breaking, since the functions were only used in the old, removed examples. Each and every moved function is still documented. They have been partly renamed, and their documentation and testing has been extended.

Changed

• Bumped and added dependencies on all 3 Project.toml files, the main one, the docs/, an the tutorials/ one.
• artificial_S2_lemniscate is available as ManoptExamples.Lemniscate and works on arbitrary manifolds now.
• artificial_S1_signal is available as ManoptExamples.artificial_S1_signal
• artificial_S1_slope_signal is available as ManoptExamples.artificial_S1_slope_signal
• artificial_S2_composite_bezier_curve is available as ManoptExamples.artificial_S2_composite_Bezier_curve
• artificial_S2_rotation_image is available as ManoptExamples.artificial_S2_rotation_image
• artificial_S2_whirl_image is available as ManoptExamples.artificial_S2_whirl_image
• artificial_S2_whirl_patch is available as ManoptExamples.artificial_S2_whirl_patch
• artificial_SAR_image is available as ManoptExamples.artificial_SAR_image
• artificial_SPD_image is available as ManoptExamples.artificial_SPD_image
• artificial_SPD_image2 is available as ManoptExamples.artificial_SPD_image2
• adjoint_differential_forward_logs is available as ManoptExamples.adjoint_differential_forward_logs
• adjoint_differential_bezier_control is available as ManoptExamples.adjoint_differential_Bezier_control_points
• BezierSegment is available as ManoptExamples.BezierSegment
• cost_acceleration_bezier is available as ManoptExamples.acceleration_Bezier
• cost_L2_acceleration_bezier is available as ManoptExamples.L2_acceleration_Bezier
• costIntrICTV12 is available as ManoptExamples.Intrinsic_infimal_convolution_TV12
• costL2TV is available as ManoptExamples.L2_Total_Variation
• costL2TV12 is available as ManoptExamples.L2_Total_Variation_1_2
• costL2TV2 is available as ManoptExamples.L2_second_order_Total_Variation
• costTV is available as ManoptExamples.Total_Variation
• costTV2 is available as ManoptExamples.second_order_Total_Variation
• de_casteljau is available as ManoptExamples.de_Casteljau
• differential_forward_logs is available as ManoptExamples.differential_forward_logs
• differential_bezier_control is available as ManoptExamples.differential_Bezier_control_points
• forward_logs is available as ManoptExamples.forward_logs
• get_bezier_degree is available as ManoptExamples.get_Bezier_degree
• get_bezier_degrees is available as ManoptExamples.get_Bezier_degrees
• get_Bezier_inner_points is available as ManoptExamples.get_Bezier_inner_points
• get_bezier_junction_tangent_vectors is available as ManoptExamples.get_Bezier_junction_tangent_vectors
• get_bezier_junctions is available as ManoptExamples.get_Bezier_junctions
• get_bezier_points is available as ManoptExamples.get_Bezier_points
• get_bezier_segments is available as ManoptExamples.get_Bezier_segments
• grad_acceleration_bezier is available as ManoptExamples.grad_acceleration_Bezier
• grad_L2_acceleration_bezier is available as ManoptExamples.grad_L2_acceleration_Bezier
• grad_Intrinsic_infimal_convolution_TV12 is available as ManoptExamples.grad_intrinsic_infimal_convolution_TV12
• grad_TV is available as ManoptExamples.grad_Total_Variation
• project_collaborative_TV is available as ManoptExamples.project_collaborative_TV
• prox_parallel_TV is available as ManoptExamples.prox_parallel_TV
• grad_TV2 is available as ManoptExamples.grad_second_order_Total_Variation
• prox_TV is available as ManoptExamples.prox_Total_Variation
• prox_TV2 is available as ManoptExamples.prox_second_order_Total_Variation

0.4.43 (November 19, 2023)

Added

• vale.sh as a CI to keep track of a consistent documentation

0.4.42 (November 6, 2023)

Added

• add Manopt.JuMP_Optimizer implementing JuMP's solver interface

0.4.41 (November 2, 2023)

Changed

• trust_regions is now more flexible and the sub solver (Steihaug-Toint tCG by default) can now be exchanged.
• adaptive_regularization_with_cubics is now more flexible as well, where it previously was a bit too much tightened to the Lanczos solver as well.
• Unified documentation notation and bumped dependencies to use DocumenterCitations 1.3

0.4.40 (October 24, 2023)

Added

• add a --help argument to docs/make.jl to document all available command line arguments
• add a --exclude-tutorials argument to docs/make.jl. This way, when quarto is not available on a computer, the docs can still be build with the tutorials not being added to the menu such that documenter does not expect them to exist.

Changed

• Bump dependencies to ManifoldsBase.jl 0.15 and Manifolds.jl 0.9
• move the ARC CG subsolver to the main package, since TangentSpace is now already available from ManifoldsBase.

0.4.39 (October 9, 2023)

Changed

• also use the pair of a retraction and the inverse retraction (see last update) to perform the relaxation within the Douglas-Rachford algorithm.

0.4.38 (October 8, 2023)

Changed

• avoid allocations when calling get_jacobian! within the Levenberg-Marquard Algorithm.

Fixed

• Fix a lot of typos in the documentation

0.4.37 (September 28, 2023)

Changed

• add more of the Riemannian Levenberg-Marquard algorithms parameters as keywords, so they can be changed on call
• generalize the internal reflection of Douglas-Rachford, such that is also works with an arbitrary pair of a reflection and an inverse reflection.

0.4.36 (September 20, 2023)

Fixed

• Fixed a bug that caused non-matrix points and vectors to fail when working with an approximate Hessian.

0.4.35 (September 14, 2023)

Added

• The access to functions of the objective is now unified and encapsulated in proper get_ functions.

0.4.34 (September 02, 2023)

Added

• an ManifoldEuclideanGradientObjective to allow the cost, gradient, and Hessian and other first or second derivative based elements to be Euclidean and converted when needed.
• a keyword objective_type=:Euclidean for all solvers, that specifies that an Objective shall be created of the new type

0.4.33 (August 24, 2023)

Added

• ConstantStepsize and DecreasingStepsize now have an additional field type::Symbol to assess whether the step-size should be relatively (to the gradient norm) or absolutely constant.

0.4.32 (August 23, 2023)

Added

• The adaptive regularization with cubics (ARC) solver.

0.4.31 (August 14, 2023)

Added

• A :Subsolver keyword in the debug= keyword argument, that activates the new DebugWhenActive to de/activate subsolver debug from the main solvers DebugEvery.

0.4.30 (August 3, 2023)

Changed

• References in the documentation are now rendered using DocumenterCitations.jl
• Asymptote export now also accepts a size in pixel instead of its default 4cm size and render can be deactivated setting it to nothing.

0.4.29 (July 12, 2023)

Fixed

• fixed a bug, where cyclic_proximal_point did not work with decorated objectives.

0.4.28 (June 24, 2023)

Changed

• max_stepsize was specialized for FixedRankManifold to follow Matlab Manopt.

0.4.27 (June 15, 2023)

Added

• The AdaptiveWNGrad stepsize is available as a new stepsize functor.

Fixed

• Levenberg-Marquardt now possesses its parameters initial_residual_values and initial_jacobian_f also as keyword arguments, such that their default initialisations can be adapted, if necessary

0.4.26 (June 11, 2023)

Added

• simplify usage of gradient descent as sub solver in the DoC solvers.
• add a get_state function
• document indicates_convergence.

0.4.25 (June 5, 2023)

Fixed

• Fixes an allocation bug in the difference of convex algorithm

0.4.24 (June 4, 2023)

Added

• another workflow that deletes old PR renderings from the docs to keep them smaller in overall size.

Changed

• bump dependencies since the extension between Manifolds.jl and ManifoldsDiff.jl has been moved to Manifolds.jl

0.4.23 (June 4, 2023)

Added

• More details on the Count and Cache tutorial

Changed

• loosen constraints slightly

0.4.22 (May 31, 2023)

Added

• A tutorial on how to implement a solver

0.4.21 (May 22, 2023)

Added

• A ManifoldCacheObjective as a decorator for objectives to cache results of calls, using LRU Caches as a weak dependency. For now this works with cost and gradient evaluations
• A ManifoldCountObjective as a decorator for objectives to enable counting of calls to for example the cost and the gradient
• adds a return_objective keyword, that switches the return of a solver to a tuple (o, s), where o is the (possibly decorated) objective, and s is the “classical” solver return (state or point). This way the counted values can be accessed and the cache can be reused.
• change solvers on the mid level (form solver(M, objective, p)) to also accept decorated objectives

Changed

• Switch all Requires weak dependencies to actual weak dependencies starting in Julia 1.9

0.4.20 (May 11, 2023)

Changed

• the default tolerances for the numerical check_ functions were loosened a bit, such that check_vector can also be changed in its tolerances.

0.4.19 (May 7, 2023)

Added

• the sub solver for trust_regions is now customizable and can now be exchanged.

Changed

• slightly changed the definitions of the solver states for ALM and EPM to be type stable

0.4.18 (May 4, 2023)

Added

• A function check_Hessian(M, f, grad_f, Hess_f) to numerically verify the (Riemannian) Hessian of a function f

0.4.17 (April 28, 2023)

Added

• A new interface of the form alg(M, objective, p0) to allow to reuse objectives without creating AbstractManoptSolverStates and calling solve!. This especially still allows for any decoration of the objective and/or the state using debug=, or record=.

Changed

• All solvers now have the initial point p as an optional parameter making it more accessible to first time users, gradient_descent(M, f, grad_f) is equivalent to gradient_descent(M, f, grad_f, rand(M))

Fixed

• Unified the framework to work on manifold where points are represented by numbers for several solvers

0.4.16 (April 18, 2023)

Fixed

• the inner products used in truncated_gradient_descent now also work thoroughly on complex matrix manifolds

0.4.15 (April 13, 2023)

Changed

• trust_regions(M, f, grad_f, hess_f, p) now has the Hessian hess_f as well as the start point p0 as an optional parameter and approximate it otherwise.
• trust_regions!(M, f, grad_f, hess_f, p) has the Hessian as an optional parameter and approximate it otherwise.

Removed

• support for ManifoldsBase.jl 0.13.x, since with the definition of copy(M,p::Number), in 0.14.4, that one is used instead of defining it ourselves.

0.4.14 (April 09, 2023)

Changed

• particle_swarm now uses much more in-place operations

Fixed

• particle_swarm used quite a few deepcopy(p) commands still, which were replaced by copy(M, p)

0.4.13 (April 06, 2023)

Added

• get_message to obtain messages from sub steps of a solver
• DebugMessages to display the new messages in debug
• safeguards in Armijo line search and L-BFGS against numerical over- and underflow that report in messages

0.4.12 (April 4, 2023)

Added

• Introduce the Difference of Convex Algorithm (DCA) difference_of_convex_algorithm(M, f, g, ∂h, p0)
• Introduce the Difference of Convex Proximal Point Algorithm (DCPPA) difference_of_convex_proximal_point(M, prox_g, grad_h, p0)
• Introduce a StopWhenGradientChangeLess stopping criterion

0.4.11 (March 27, 2023)

Changed

• adapt tolerances in tests to the speed/accuracy optimized distance on the sphere in Manifolds.jl (part II)

0.4.10 (March 26, 2023)

Changed

• adapt tolerances in tests to the speed/accuracy optimized distance on the sphere in Manifolds.jl

0.4.9 (March 3, 2023)

Added

• introduce a wrapper that allows line searches from LineSearches.jl to be used within Manopt.jl, introduce the manoptjl.org/stable/extensions/ page to explain the details.

0.4.8 (February 21, 2023)

Added

• a status_summary that displays the main parameters within several structures of Manopt, most prominently a solver state

Changed

• Improved storage performance by introducing separate named tuples for points and vectors
• changed the show methods of AbstractManoptSolverStates to display their state_summary
• Move tutorials to be rendered with Quarto into the documentation.

0.4.7 (February 14, 2023)

Changed

• Bump [compat] entry of ManifoldDiff to also include 0.3

0.4.6 (February 3, 2023)

Fixed

• Fixed a few stopping criteria even indicated to stop before the algorithm started.

0.4.5 (January 24, 2023)

Changed

• the new default functions that include p are used where possible
• a first step towards faster storage handling

0.4.4 (January 20, 2023)

Added

• Introduce ConjugateGradientBealeRestart to allow CG restarts using Beale‘s rule

Fixed

• fix a typo in HestenesStiefelCoefficient

0.4.3 (January 17, 2023)

Fixed

• the CG coefficient β can now be complex
• fix a bug in grad_distance

0.4.2 (January 16, 2023)

Changed

• the usage of inner in line search methods, such that they work well with complex manifolds as well

0.4.1 (January 15, 2023)

Fixed

• a max_stepsize per manifold to avoid leaving the injectivity radius, which it also defaults to

0.4.0 (January 10, 2023)

This is the first version with an actual Changelog entry

Added

• Dependency on ManifoldDiff.jl and a start of moving actual derivatives, differentials, and gradients there.
• AbstractManifoldObjective to store the objective within the AbstractManoptProblem
• Introduce a CostGrad structure to store a function that computes the cost and gradient within one function.
• started a changelog.md to thoroughly keep track of changes

Changed

• AbstractManoptProblem replaces Problem
• the problem now contains an AbstractManifoldObjective
• AbstractManoptSolverState replaces Options
• random_point(M) is replaced by rand(M) from ManifoldsBase.jl
• random_tangent(M, p) is replaced by rand(M; vector_at=p)

0.3.0 (March 9, 2021)

0.2.0 (April 19, 2020)

0.1.0 (June 3, 2019)

First public release, first registered and announced version of Manopt.jl. This version still also included what is now Manifolds.jl.

The first commit that started Manopt.jl was done on November 25, 2016.