GitHub

Extensions

LineSearches.jl

Manopt can be used with line search algorithms implemented in LineSearches.jl. This can be illustrated by the following example of optimizing Rosenbrock function constrained to the unit sphere.

using Manopt, Manifolds, LineSearches

# define objective function and its gradient
p = [1.0, 100.0]
function rosenbrock(::AbstractManifold, x)
    val = zero(eltype(x))
    for i in 1:(length(x) - 1)
        val += (p[1] - x[i])^2 + p[2] * (x[i + 1] - x[i]^2)^2
    end
    return val
end
function rosenbrock_grad!(M::AbstractManifold, storage, x)
    storage .= 0.0
    for i in 1:(length(x) - 1)
        storage[i] += -2.0 * (p[1] - x[i]) - 4.0 * p[2] * (x[i + 1] - x[i]^2) * x[i]
        storage[i + 1] += 2.0 * p[2] * (x[i + 1] - x[i]^2)
    end
    project!(M, storage, x, storage)
    return storage
end
# define constraint
n_dims = 5
M = Manifolds.Sphere(n_dims)
# set initial point
x0 = vcat(zeros(n_dims - 1), 1.0)
# use LineSearches.jl HagerZhang method with Manopt.jl quasiNewton solver
ls_hz = Manopt.LineSearchesStepsize(M, LineSearches.HagerZhang())
x_opt = quasi_Newton(
    M,
    rosenbrock,
    rosenbrock_grad!,
    x0;
    stepsize=ls_hz,
    evaluation=InplaceEvaluation(),
    stopping_criterion=StopAfterIteration(1000) | StopWhenGradientNormLess(1e-6),
    return_state=true,
)
# Solver state for `Manopt.jl`s Quasi Newton Method
After 15 iterations

## Parameters
* direction update:        limited memory InverseBFGS (size 5) initial scaling 1.0and ParallelTransport() as vector transport.
* retraction method:       StabilizedRetraction()
* vector transport method: ParallelTransport()

## Stepsize
A step size wrapper for LineSearches.jl
(last step size: 1.0)

## Parameters
* line search:              HagerZhang{Float64, Base.RefValue{Bool}}(0.1, 0.9, Inf, 5.0, 1.0e-6, 0.6666666666666666, 50, 0.1, 0, Base.RefValue{Bool}(false), nothing, false)
* initial guess:            Manopt.ConstantInitialGuess{Float64}(1.0)
* retraction method:        StabilizedRetraction()
* vector transport method:  ParallelTransport()


## Stopping criterion
Stop when _one_ of the following are fulfilled:
  * stopped after 1000 iterations:  not reached
  * |grad f| < 1.0e-6:  reached
Overall: reached
This indicates convergence: No

In general this defines the following new stepsize with helper functions for setting and getting the maximum step size:

Manopt.LineSearchesStepsize — Type
LineSearchesStepsize <: Stepsize

Wrapper for line searches available in the LineSearches.jl library.

Constructors

LineSearchesStepsize(M::AbstractManifold, linesearch; kwargs...
LineSearchesStepsize(
    linesearch; retraction_method=ExponentialRetraction(), vector_transport_method=ParallelTransport(),
)

Wrap linesearch (for example HagerZhang or MoreThuente). The initial step selection from Linesearches.jl is not yet supported and initial_guess is always used (by default ConstantInitialGuess).

Keyword Arguments

source
Manopt.linesearches_set_max_alpha — Function
linesearches_set_max_alpha(ls, max_alpha::Real)

Set the maximum step size for LineSearches.jl line search ls to max_alpha. Return a new line search object with the updated maximum step size.

source

Manifolds.jl

Loading Manifolds.jl introduces the following additional functions

Manopt.max_stepsize — Method
max_stepsize(M::FixedRankMatrices, p)

Return a reasonable guess of maximum step size on FixedRankMatrices following the choice of typical distance in Matlab Manopt, the dimension of M. See this note

source
Manopt.max_stepsize — Method
max_stepsize(M::Hyperrectangle, p)

The default maximum stepsize for Hyperrectangle manifold with corners is maximum of distances from p to each boundary.

source
Manopt.max_stepsize — Method
max_stepsize(M::TangentBundle, p)

Tangent bundle has injectivity radius of either infinity (for flat manifolds) or 0 (for non-flat manifolds). This makes a guess of what a reasonable maximum stepsize on a tangent bundle might be.

source
ManifoldsBase.mid_point — Function
mid_point(M, p, q, x)
mid_point!(M, y, p, q, x)

Compute the mid point between p and q. If there is more than one mid point of (not necessarily minimizing) geodesics (for example on the sphere), the one nearest to x is returned (in place of y).

source

Internally, Manopt.jl provides the two additional functions to choose some Euclidean space when needed as

Manopt.Rn — Function
Rn(args; kwargs...)
Rn(s::Symbol=:Manifolds, args; kwargs...)

A small internal helper function to choose a Euclidean space. By default, this uses the DefaultManifold unless you load a more advanced Euclidean space like Euclidean from Manifolds.jl

source
Manopt.Rn_default — Function
Rn_default()

Specify a default value to dispatch Rn on. This default is set to Manifolds, indicating, that when this package is loaded, it is the preferred package to ask for a vector space space.

The default within Manopt.jl is to use the DefaultManifold from ManifoldsBase.jl. If you load Manifolds.jl this switches to using Euclidean.

source