Alternating Gradient Descent

alternating_gradient_descent(M::ProductManifold, f, grad_f, p)

perform an alternating gradient descent

Input

• M – the product manifold $\mathcal M = \mathcal M_1 × \mathcal M_2 × ⋯ ×\mathcal M_n$
• f – the objective function (cost) defined on M.
• grad_f – a gradient, that can be of two cases
  • is a single function returning a ProductRepr or
  • is a vector functions each returning a component part of the whole gradient
• p – an initial value $p_0 ∈ \mathcal M$

Optional

• evaluation – (AllocatingEvaluation) specify whether the gradient(s) works by allocation (default) form gradF(M, x) or InplaceEvaluation in place, i.e. is of the form gradF!(M, X, x) (elementwise).
• evaluation_order – (:Linear) – whether to use a randomly permuted sequence (:FixedRandom), a per cycle permuted sequence (:Random) or the default :Linear one.
• inner_iterations– (5) how many gradient steps to take in a component before alternating to the next
• stopping_criterion (StopAfterIteration(1000))– a StoppingCriterion
• stepsize (ArmijoLinesearch()) a Stepsize
• order - ([1:n]) the initial permutation, where n is the number of gradients in gradF.
• retraction_method – (default_retraction_method(M, typeof(p))) a retraction(M, p, X) to use.

Output

usually the obtained (approximate) minimizer, see get_solver_return for details

alternating_gradient_descent!(M::ProductManifold, f, grad_f, p)

perform a alternating gradient descent in place of p.

Input

• M a product manifold $\mathcal M$
• f – the objective functioN (cost)
• grad_f – a gradient function, that either returns a vector of the subgradients or is a vector of gradients
• p – an initial value $p_0 ∈ \mathcal M$

for all optional parameters, see alternating_gradient_descent.

AlternatingGradientDescentState <: AbstractGradientDescentSolverState

Store the fields for an alternating gradient descent algorithm, see also alternating_gradient_descent.

Fields

• direction (AlternatingGradient(zero_vector(M, x)) a DirectionUpdateRule
• evaluation_order – (:Linear) – whether
• inner_iterations– (5) how many gradient steps to take in a component before alternating to the next to use a randomly permuted sequence (:FixedRandom), a per cycle newly permuted sequence (:Random) or the default :Linear evaluation order.
• order the current permutation
• retraction_method – (default_retraction_method(M, typeof(p))) a retraction(M,x,ξ) to use.
• stepsize (ConstantStepsize(M)) a Stepsize
• stopping_criterion (StopAfterIteration(1000))– a StoppingCriterion
• p the current iterate
• X (zero_vector(M,p)) the current gradient tangent vector
• k, ì` internal counters for the outer and inner iterations, respectively.

Constructors

AlternatingGradientDescentState(M, p; kwargs...)

Generate the options for point p and and where inner_iterations, order_type, order, retraction_method, stopping_criterion, and stepsize` are keyword arguments

AlternatingGradient <: DirectionUpdateRule

The default gradient processor, which just evaluates the (alternating) gradient on one of the components