Alternating gradient descent

alternating_gradient_descent(M::ProductManifold, f, grad_f, p=rand(M))
alternating_gradient_descent(M::ProductManifold, ago::ManifoldAlternatingGradientObjective, p)
alternating_gradient_descent!(M::ProductManifold, f, grad_f, p)
alternating_gradient_descent!(M::ProductManifold, ago::ManifoldAlternatingGradientObjective, p)

perform an alternating gradient descent. This can be done in-place of the start point p.

Input

• M::AbstractManifold: a Riemannian manifold $\mathcal{M}$
• f: a cost function $f: \mathcal{M}→ ℝ$ implemented as (M, p) -> v
• grad_f: a gradient, that can be of two cases
  • is a single function returning an ArrayPartition from RecursiveArrayTools.jl or
  • is a vector of functions, each returning a component part of the whole gradient
• p::P: a point on the manifold $\mathcal{M}$

Keyword arguments

• callbacks::D = Dict{Symbol,Function}(): provided callback functions given either as a single function (symbol, problem, state, k) called in every hook or as a (vector of) pairs :hook => function, which are processed by process_callbacks_arg. As key you can either pass single symbol or an array of symbols to indicate a callback should be added in multiple places
• evaluation::AbstractEvaluationType=AllocatingEvaluation(): specify whether the functions that return an array, for example a point or a tangent vector, work by allocating their result (AllocatingEvaluation) or whether they modify their input argument to return the result therein (InplaceEvaluation). Since usually the first argument is the manifold, the modified argument is the second.
• order_type=:Linear: whether to use a randomly permuted sequence (:FixedRandom), a per cycle newly permuted sequence (:Random) or the default :Linear evaluation order.
• inner_iterations=5: how many gradient steps to take in a component before alternating to the next
• stopping_criterion::StoppingCriterion=StopAfterIteration(100)|StopWhenGradientNormLess(1.0e-9): a functor indicating that the stopping criterion is fulfilled
• stepsize::Stepsize=default_stepsize(M,AlternatingGradientDescentState; retraction_method=retraction_method): a functor inheriting from Stepsize to determine a step size
• order=collect(1:n): the initial permutation, where n is the number of gradients in grad_f.
• retraction_method::AbstractRetractionMethod=default_retraction_method(M, typeof(p)): a retraction $\operatorname{retr}$ to use, see the section on retractions

Output

usually the obtained (approximate) minimizer, see get_solver_return for details

alternating_gradient_descent(M::ProductManifold, f, grad_f, p=rand(M))
alternating_gradient_descent(M::ProductManifold, ago::ManifoldAlternatingGradientObjective, p)
alternating_gradient_descent!(M::ProductManifold, f, grad_f, p)
alternating_gradient_descent!(M::ProductManifold, ago::ManifoldAlternatingGradientObjective, p)

perform an alternating gradient descent. This can be done in-place of the start point p.

Input

• M::AbstractManifold: a Riemannian manifold $\mathcal{M}$
• f: a cost function $f: \mathcal{M}→ ℝ$ implemented as (M, p) -> v
• grad_f: a gradient, that can be of two cases
  • is a single function returning an ArrayPartition from RecursiveArrayTools.jl or
  • is a vector of functions, each returning a component part of the whole gradient
• p::P: a point on the manifold $\mathcal{M}$

Keyword arguments

• callbacks::D = Dict{Symbol,Function}(): provided callback functions given either as a single function (symbol, problem, state, k) called in every hook or as a (vector of) pairs :hook => function, which are processed by process_callbacks_arg. As key you can either pass single symbol or an array of symbols to indicate a callback should be added in multiple places
• evaluation::AbstractEvaluationType=AllocatingEvaluation(): specify whether the functions that return an array, for example a point or a tangent vector, work by allocating their result (AllocatingEvaluation) or whether they modify their input argument to return the result therein (InplaceEvaluation). Since usually the first argument is the manifold, the modified argument is the second.
• order_type=:Linear: whether to use a randomly permuted sequence (:FixedRandom), a per cycle newly permuted sequence (:Random) or the default :Linear evaluation order.
• inner_iterations=5: how many gradient steps to take in a component before alternating to the next
• stopping_criterion::StoppingCriterion=StopAfterIteration(100)|StopWhenGradientNormLess(1.0e-9): a functor indicating that the stopping criterion is fulfilled
• stepsize::Stepsize=default_stepsize(M,AlternatingGradientDescentState; retraction_method=retraction_method): a functor inheriting from Stepsize to determine a step size
• order=collect(1:n): the initial permutation, where n is the number of gradients in grad_f.
• retraction_method::AbstractRetractionMethod=default_retraction_method(M, typeof(p)): a retraction $\operatorname{retr}$ to use, see the section on retractions

Output

usually the obtained (approximate) minimizer, see get_solver_return for details

AlternatingGradientDescentState <: AbstractGradientSolverState

Store the fields for an alternating gradient descent algorithm, see also alternating_gradient_descent.

Fields

• callbacks::D: provided callback functions given as a dictionary with symbols as keys
• direction::DirectionUpdateRule
• order_type::Symbol: whether to use a randomly permuted sequence (:FixedRandom), a per cycle newly permuted sequence (:Random) or the default :Linear evaluation order.
• inner_iterations: how many gradient steps to take in a component before alternating to the next
• order: the current permutation
• retraction_method::AbstractRetractionMethod: a retraction $\operatorname{retr}$ to use, see the section on retractions
• stepsize::Stepsize: a functor inheriting from Stepsize to determine a step size
• stop::StoppingCriterion: a functor indicating that the stopping criterion is fulfilled
• p::P: a point on the manifold $\mathcal{M}$ storing the current iterate
• X::T: a tangent vector at the point $p$ on the manifold $\mathcal{M}$ storing the gradient at the current iterate
• k, i: internal counters for the current component of the order and the inner iterations within that component, respectively.

Constructors

AlternatingGradientDescentState(M::AbstractManifold; kwargs...)

Keyword arguments

• inner_iterations=5
• p::P =rand(M): a point on the manifold $\mathcal{M}$
• callbacks::D = Dict{Symbol,Function}(): provided callback functions given as a dictionary with symbols as keys
• order_type::Symbol=:Linear
• order::Vector{<:Int}=Int[]
• stopping_criterion::StoppingCriterion=StopAfterIteration(1000): a functor indicating that the stopping criterion is fulfilled
• retraction_method::AbstractRetractionMethod=default_retraction_method(M, typeof(p)): a retraction $\operatorname{retr}$ to use, see the section on retractions
• stepsize::Stepsize=default_stepsize(M, AlternatingGradientDescentState; retraction_method=retraction_method): a functor inheriting from Stepsize to determine a step size
• X::T =zero_vector(M, p): a tangent vector at the point $p$ on the manifold $\mathcal{M}$

Generate the state for point p and where inner_iterations, order_type, order, retraction_method, stopping_criterion, and stepsize are keyword arguments.

For internal use, there also exists a constructor solely having the fields as keyword arguments, but then all of them are mandatory.

AlternatingGradient(; kwargs...)
AlternatingGradient(M::AbstractManifold; kwargs...)

Specify that a gradient based method should only update parts of the gradient in order to do an alternating gradient descent.

Keyword arguments

• X::T =zero_vector(M, p): a tangent vector at the point $p$ on the manifold $\mathcal{M}$
• p::P =rand(M): a point on the manifold $\mathcal{M}$ to specify the initial value

AlternatingGradientRule <: AbstractGradientGroupDirectionRule

The direction processor to alternate the gradient directions.

Create a functor (problem, state, k) -> (s, X) to evaluate the alternating gradient, that is alternating between the components of the gradient and has a field for partial evaluation of the gradient in-place.

Fields

• X::T: a tangent vector at the point $p$ on the manifold $\mathcal{M}$

Constructor

AlternatingGradientRule(M::AbstractManifold; p=rand(M), X=zero_vector(M, p))

Initialize the alternating gradient processor with tangent vector type of X, where both M and p are just help variables.

See also

alternating_gradient_descent, AlternatingGradient