Extensions

LineSearches.jl

Manopt.jl can be used with line search algorithms implemented in LineSearches.jl. This can be illustrated by the following example of optimizing the Rosenbrock function constrained to the unit sphere.

using Manopt, Manifolds, LineSearches# define objective function and its gradientp = [1.0, 100.0]function rosenbrock(::AbstractManifold, x)    val = zero(eltype(x))    for i in 1:(length(x) - 1)        val += (p[1] - x[i])^2 + p[2] * (x[i + 1] - x[i]^2)^2    end    return valendfunction rosenbrock_grad!(M::AbstractManifold, storage, x)    storage .= 0.0    for i in 1:(length(x) - 1)        storage[i] += -2.0 * (p[1] - x[i]) - 4.0 * p[2] * (x[i + 1] - x[i]^2) * x[i]        storage[i + 1] += 2.0 * p[2] * (x[i + 1] - x[i]^2)    end    project!(M, storage, x, storage)    return storageend# define constraintn_dims = 5M = Manifolds.Sphere(n_dims)# set initial pointx0 = vcat(zeros(n_dims - 1), 1.0)# use LineSearches.jl HagerZhang method with Manopt.jl quasiNewton solverls_hz = Manopt.LineSearchesStepsize(M, LineSearches.HagerZhang())x_opt = quasi_Newton(    M,    rosenbrock,    rosenbrock_grad!,    x0;    stepsize=ls_hz,    evaluation=InplaceEvaluation(),    stopping_criterion=StopAfterIteration(1000) | StopWhenGradientNormLess(1e-6),    return_state=true,)

# Solver state for `Manopt.jl`s Quasi Newton Method
After 15 iterations

## Parameters
* direction update:        limited memory InverseBFGS (size 5) initial scaling 1.0and ParallelTransport() as vector transport.
* retraction method:       StabilizedRetraction()
* vector transport method: ParallelTransport()

## Stepsize
A line search step size wrapper for LineSearches.jl
(last step size: 1.0)

## Parameters
* line search:              HagerZhang{Float64, Base.RefValue{Bool}}(0.1, 0.9, Inf, 5.0, 1.0e-6, 0.6666666666666666, 50, 0.1, 0, Base.RefValue{Bool}(false), nothing, false)
* initial guess:            Manopt.ConstantInitialGuess{Float64}(1.0)
* retraction method:        StabilizedRetraction()
* vector transport method:  ParallelTransport()


## Stopping criterion
Stop when _one_ of the following is fulfilled:
  *   stopped after 1000 iterations:  not reached
  *   |grad f| < 1.0e-6:  reached
Overall: reached
This indicates convergence: No

In general this defines the following new stepsize with helper functions for setting and getting the maximum step size:

Manopt.LineSearchesStepsize — Type

LineSearchesStepsize <: Stepsize

Wrapper for line searches available in the LineSearches.jl library.

Constructors

LineSearchesStepsize(M::AbstractManifold, linesearch; kwargs...)LineSearchesStepsize(linesearch; kwargs...)

Wrap linesearch (for example HagerZhang or MoreThuente). The initial step selection from LineSearches.jl is not yet supported and initial_guess is always used instead.

Keyword Arguments

initial_guess=ConstantInitialGuess(): the initial guess for the step size to start the line search from.
last_stepsize=NaN: the initial value to store as the last step size computed.
retraction_method::AbstractRetractionMethod=default_retraction_method(M, typeof(p)): a retraction $\operatorname{retr}$ to use, see the section on retractions
vector_transport_method::AbstractVectorTransportMethod=default_vector_transport_method(M, typeof(p)): a vector transport $\mathcal T_{⋅←⋅}$ to use, see the section on vector transports

Without the manifold M as a first argument, retraction_method defaults to ExponentialRetraction() and vector_transport_method to ParallelTransport().

source

Manopt.linesearches_get_max_alpha — Function

linesearches_get_max_alpha(ls)

Get the maximum step size for LineSearches.jl line search ls.

source

Manopt.linesearches_set_max_alpha — Function

linesearches_set_max_alpha(ls, max_alpha::Real)

Set the maximum step size for LineSearches.jl line search ls to max_alpha. Return a new line search object with the updated maximum step size.

source

Manifolds.jl

Loading Manifolds.jl introduces the following additional functions:

Manopt.max_stepsize — Method

max_stepsize(M::FixedRankMatrices, p)

Return a reasonable guess of maximum step size on FixedRankMatrices following the choice of typical distance in Matlab Manopt, the dimension of M. See this note

source

Manopt.max_stepsize — Method

max_stepsize(M::Hyperrectangle, p)

The default maximum stepsize for Hyperrectangle manifold with corners is maximum of distances from p to each boundary.

source

Manopt.max_stepsize — Method

max_stepsize(M::TangentBundle, p)

Tangent bundle has injectivity radius of either infinity (for flat manifolds) or 0 (for non-flat manifolds). This makes a guess of what a reasonable maximum stepsize on a tangent bundle might be.

source

ManifoldsBase.mid_point — Function

mid_point(M, p, q, x)
mid_point!(M, y, p, q, x)

Compute the mid point between p and q. If there is more than one mid point of (not necessarily minimizing) geodesics (for example on the sphere), the one nearest to x is returned (in place of y).

source

Internally, Manopt.jl provides the following two additional functions to choose some Euclidean space when needed:

Manopt.Rn — Function

Rn(args; kwargs...)
Rn(s::Symbol=:Manifolds, args; kwargs...)

A small internal helper function to choose a Euclidean space. By default, this uses the DefaultManifold unless you load a more advanced Euclidean space like Euclidean from Manifolds.jl.

source

Manopt.Rn_default — Function

Rn_default()

Specify a default value to dispatch Rn on. This default is set to Manifolds, indicating that, when this package is loaded, it is the preferred package to ask for a vector space.

The default within Manopt.jl is to use the DefaultManifold from ManifoldsBase.jl. If you load Manifolds.jl this switches to using Euclidean.

source