Extensions

LineSearches.jl

Manopt can be used with line search algorithms implemented in LineSearches.jl. This can be illustrated by the following example of optimizing Rosenbrock function constrained to the unit sphere.

using Manopt, Manifolds, LineSearches

# define objective function and its gradient
p = [1.0, 100.0]
function rosenbrock(::AbstractManifold, x)
    val = zero(eltype(x))
    for i in 1:(length(x) - 1)
        val += (p[1] - x[i])^2 + p[2] * (x[i + 1] - x[i]^2)^2
    end
    return val
end
function rosenbrock_grad!(M::AbstractManifold, storage, x)
    storage .= 0.0
    for i in 1:(length(x) - 1)
        storage[i] += -2.0 * (p[1] - x[i]) - 4.0 * p[2] * (x[i + 1] - x[i]^2) * x[i]
        storage[i + 1] += 2.0 * p[2] * (x[i + 1] - x[i]^2)
    end
    project!(M, storage, x, storage)
    return storage
end
# define constraint
n_dims = 5
M = Manifolds.Sphere(n_dims)
# set initial point
x0 = vcat(zeros(n_dims - 1), 1.0)
# use LineSearches.jl HagerZhang method with Manopt.jl quasiNewton solver
ls_hz = Manopt.LineSearchesStepsize(M, LineSearches.HagerZhang())
x_opt = quasi_Newton(
    M,
    rosenbrock,
    rosenbrock_grad!,
    x0;
    stepsize=ls_hz,
    evaluation=InplaceEvaluation(),
    stopping_criterion=StopAfterIteration(1000) | StopWhenGradientNormLess(1e-6),
    return_state=true,
)

# Solver state for `Manopt.jl`s Quasi Newton Method
After 10 iterations

## Parameters
* direction update:        limited memory InverseBFGS (size 5) initial scaling 1.0and ParallelTransport() as vector transport.
* retraction method:       ExponentialRetraction()
* vector transport method: ParallelTransport()

## Stepsize
LineSearchesStepsize(HagerZhang{Float64, Base.RefValue{Bool}}
  delta: Float64 0.1
  sigma: Float64 0.9
  alphamax: Float64 Inf
  rho: Float64 5.0
  epsilon: Float64 1.0e-6
  gamma: Float64 0.66
  linesearchmax: Int64 50
  psi3: Float64 0.1
  display: Int64 0
  mayterminate: Base.RefValue{Bool}
  cache: Nothing nothing
  check_flatness: Bool false
; retraction_method=ExponentialRetraction(), vector_transport_method=ParallelTransport())

## Stopping criterion

Stop When _one_ of the following are fulfilled:
  * Max Iteration 1000:	not reached
  * |grad f| < 1.0e-6: reached
Overall: reached
This indicates convergence: Yes

In general this defines the following new stepsize

Manopt.LineSearchesStepsize — Type

LineSearchesStepsize <: Stepsize

Wrapper for line searches available in the LineSearches.jl library.

Constructors

LineSearchesStepsize(M::AbstractManifold, linesearch; kwargs...
LineSearchesStepsize(
    linesearch;
    retraction_method=ExponentialRetraction(),
    vector_transport_method=ParallelTransport(),
)

Wrap linesearch (for example HagerZhang or MoreThuente). The initial step selection from Linesearches.jl is not yet supported and the value 1.0 is used.

Keyword Arguments

retraction_method=default_retraction_method(M, typeof(p)): a retraction $\operatorname{retr}$ to use, see the section on retractions
vector_transport_method=default_vector_transport_method(M, typeof(p)): a vector transport $\mathcal T_{⋅←⋅}$ to use, see the section on vector transports

Extensions

LineSearches.jl

Manifolds.jl

JuMP.jl

Internal functions

Internal wrappers and their functions