GitHub

Extensions

LineSearches.jl

Manopt.jl can be used with line search algorithms implemented in LineSearches.jl. This can be illustrated by the following example of optimizing the Rosenbrock function constrained to the unit sphere.

using Manopt, Manifolds, LineSearches# define objective function and its gradienta, b = 1.0, 100.0function rosenbrock(::AbstractManifold, p)    val = zero(eltype(p))    for i in 1:(length(p) - 1)        val += (a - p[i])^2 + b * (p[i + 1] - p[i]^2)^2    end    return valendfunction rosenbrock_grad!(M::AbstractManifold, X, p)    X .= 0.0    for i in 1:(length(p) - 1)        X[i] += -2.0 * (a - p[i]) - 4.0 * b * (p[i + 1] - p[i]^2) * p[i]        X[i + 1] += 2.0 * b * (p[i + 1] - p[i]^2)    end    project!(M, X, p, X)    return Xend# define constraintn_dims = 5M = Manifolds.Sphere(n_dims)# set initial pointp0 = vcat(zeros(n_dims), 1.0)# use LineSearches.jl HagerZhang method with Manopt.jl quasi_Newton solverls_hz = Manopt.LineSearchesStepsize(M, LineSearches.HagerZhang())p_opt = quasi_Newton(    M,    rosenbrock,    rosenbrock_grad!,    p0;    stepsize=ls_hz,    evaluation=InplaceEvaluation(),    stopping_criterion=StopAfterIteration(1000) | StopWhenGradientNormLess(1e-6),    return_state=true,)
# Solver state for `Manopt.jl`s Quasi Newton Method
After 20 iterations

## Parameters
* direction update:        A limited memory InverseBFGS direction update of memory size 5.
* retraction method:       StabilizedRetraction()
* vector transport method: ParallelTransport()

## Stepsize
A line search step size wrapper for LineSearches.jl
(last step size: 1.0)

### Parameters
* line search:              HagerZhang{Float64, Base.RefValue{Bool}}(0.1, 0.9, Inf, 5.0, 1.0e-6, 0.6666666666666666, 50, 0.1, 0, Base.RefValue{Bool}(false), nothing, false)
* initial guess:            Manopt.ConstantInitialGuess{Float64}(1.0)
* retraction method:        StabilizedRetraction()
* vector transport method:  ParallelTransport()


## Stopping criterion
Stop when _one_ of the following is fulfilled:
  *   stopped after 1000 iterations:  not reached
  *   |grad f| < 1.0e-6:  reached
Overall: reached
The algorithm converged: Yes

In general this defines the following new step size with helper functions for setting and getting the maximum step size:

Manopt.LineSearchesStepsize — Type
LineSearchesStepsize <: Stepsize

Wrapper for line searches available in the LineSearches.jl library.

Constructors

LineSearchesStepsize(M::AbstractManifold, linesearch; kwargs...)LineSearchesStepsize(linesearch; kwargs...)

Wrap linesearch (for example HagerZhang or MoreThuente). The initial step selection from LineSearches.jl is not yet supported and initial_guess is always used instead.

Keyword Arguments

Without the manifold M as a first argument, retraction_method defaults to ExponentialRetraction() and vector_transport_method to ParallelTransport().

source
Manopt.linesearches_set_max_alpha — Function
linesearches_set_max_alpha(ls, max_alpha::Real)

Set the maximum step size for LineSearches.jl line search ls to max_alpha. Return a new line search object with the updated maximum step size.

source

LRUCache.jl

Loading LRUCache.jl provides the cache used by the cache=(:LRU, ...) keyword of decorate_objective!:

Manopt.init_caches — Method
init_caches(M::AbstractManifold, caches, T::Type{LRU}; kwargs...)

Given a vector of symbols caches, this function sets up the NamedTuple of caches, where T is the type of cache to use.

Keyword arguments

  • p=rand(M): a point on a manifold, to both infer its type for keys and initialize caches
  • value=0.0: a value both typing and initializing number-caches, the default is for (Float) values like the cost.
  • X=zero_vector(M, p): a tangent vector at p to both type and initialize tangent vector caches
  • cache_size=10: a default cache size to use
  • cache_sizes=Dict{Symbol,Int}(): a dictionary of sizes for the caches to specify different (non-default) sizes
source
init_caches(M::AbstractManifold, caches, T; kwargs...)

Given a vector of symbols caches, this function sets up the NamedTuple of caches for points/vectors on M, where T is the type of cache to use.

source

Manifolds.jl

Loading Manifolds.jl introduces the following additional functions:

Manopt.default_vector_norm — Method
default_vector_norm(::Euclidean, p, X)

Following [HZ06b], the norm of a tangent vector X at p used within the Hager-Zhang initial guess on Euclidean is the Inf norm of X.

source
Manopt.max_stepsize — Method
max_stepsize(M::FixedRankMatrices, p)

Return a reasonable guess of maximum step size on FixedRankMatrices following the choice of typical distance in Matlab Manopt, the dimension of M. See this note

source
Manopt.max_stepsize — Method
max_stepsize(M::Hyperrectangle, p)

The default maximum stepsize for Hyperrectangle manifold with corners is maximum of distances from p to each boundary.

source
Manopt.max_stepsize — Method
max_stepsize(::SymmetricPositiveDefinite, p)

Return the maximum stepsize on the SymmetricPositiveDefinite manifold. The injectivity radius of SymmetricPositiveDefinite is infinite, but we return the logarithm of the maximum floating-point number to avoid numerical issues.

source
Manopt.max_stepsize — Method
max_stepsize(M::TangentBundle, p)

Tangent bundle has injectivity radius of either infinity (for flat manifolds) or 0 (for non-flat manifolds). This makes a guess of what a reasonable maximum stepsize on a tangent bundle might be.

source
ManifoldsBase.mid_point — Function
mid_point(M, p, q, x)
mid_point!(M, y, p, q, x)

Compute the mid point between p and q. If there is more than one mid point of (not necessarily minimizing) geodesics (for example on the sphere), the one nearest to x is returned (in place of y).

source

Internally, Manopt.jl provides the following two additional functions to choose some Euclidean space when needed:

Manopt.Rn — Function
Rn(args...; kwargs...)
Rn(::Val{T}, args...; kwargs...)

A small internal helper function to choose a Euclidean space. By default, this uses the DefaultManifold unless you load a more advanced Euclidean space like Euclidean from Manifolds.jl.

source
Manopt.Rn_default — Function
Rn_default()

Specify a default value to dispatch Rn on. This default is set to Manifolds, indicating that, when this package is loaded, it is the preferred package to ask for a vector space.

The default within Manopt.jl is to use the DefaultManifold from ManifoldsBase.jl. If you load Manifolds.jl this switches to using Euclidean.

source

RipQP.jl and QuadraticModels.jl

Loading both provides the default sub solvers convex_bundle_method_subsolver, proximal_bundle_method_subsolver and gradient_sampling_subsolver.

RecursiveArrayTools.jl

Loading RecursiveArrayTools.jl provides the alternating gradient descent solver on a ProductManifold as well as the following two ways to evaluate the gradient of a ManifoldAlternatingGradientObjective.