Stochastic gradient descent

stochastic_gradient_descent(M, grad_f, p; kwargs...)
stochastic_gradient_descent(M, msgo, p; kwargs...)

perform a stochastic gradient descent

Input

• M: a manifold $\mathcal M$
• grad_f: a gradient function, that either returns a vector of the subgradients or is a vector of gradients
• p: an initial value $x ∈ \mathcal M$

alternatively to the gradient you can provide an ManifoldStochasticGradientObjective msgo, then using the cost= keyword does not have any effect since if so, the cost is already within the objective.

Optional

• cost: (missing) you can provide a cost function for example to track the function value
• evaluation: (AllocatingEvaluation) specify whether the gradients works by allocation (default) form gradF(M, x) or InplaceEvaluation in place of the form gradF!(M, X, x) (elementwise).
• evaluation_order: (:Random) specify whether to use a randomly permuted sequence (:FixedRandom), a per cycle permuted sequence (:Linear) or the default :Random one.
• stopping_criterion: (StopAfterIteration(1000)) a StoppingCriterion
• stepsize: (ConstantStepsize(1.0)) a Stepsize
• order_type: (:RandomOder) a type of ordering of gradient evaluations. Possible values are :RandomOrder, a :FixedPermutation, :LinearOrder
• order: ([1:n]) the initial permutation, where n is the number of gradients in gradF.
• retraction_method: (default_retraction_method(M, typeof(p))) a retraction to use.

Output

the obtained (approximate) minimizer $p^*$, see get_solver_return for details

stochastic_gradient_descent!(M, grad_f, p)
stochastic_gradient_descent!(M, msgo, p)

perform a stochastic gradient descent in place of p.

Input

• M: a manifold $\mathcal M$
• grad_f: a gradient function, that either returns a vector of the subgradients or is a vector of gradients
• p: an initial value $p ∈ \mathcal M$

Alternatively to the gradient you can provide an ManifoldStochasticGradientObjective msgo, then using the cost= keyword does not have any effect since if so, the cost is already within the objective.

for all optional parameters, see stochastic_gradient_descent.

StochasticGradientDescentState <: AbstractGradientDescentSolverState

Store the following fields for a default stochastic gradient descent algorithm, see also ManifoldStochasticGradientObjective and stochastic_gradient_descent.

Fields

• p: the current iterate
• direction: (StochasticGradient) a direction update to use
• stopping_criterion: (StopAfterIteration(1000)) a StoppingCriterion
• stepsize: (ConstantStepsize(1.0)) a Stepsize
• evaluation_order: (:Random) specify whether to use a randomly permuted sequence (:FixedRandom), a per cycle permuted sequence (:Linear) or the default :Random one.
• order: the current permutation
• retraction_method: (default_retraction_method(M, typeof(p))) a retraction(M, p, X) to use.

Constructor

StochasticGradientDescentState(M, p)

Create a StochasticGradientDescentState with start point p. all other fields are optional keyword arguments, and the defaults are taken from M.

AbstractStochasticGradientDescentSolverState <: AbstractManoptSolverState

A generic type for all options related to stochastic gradient descent methods

StochasticGradient <: AbstractGradientGroupProcessor

The default gradient processor, which just evaluates the (stochastic) gradient or a subset thereof.

Constructor

StochasticGradient(M::AbstractManifold; p=rand(M), X=zero_vector(M, p))

Initialize the stochastic Gradient processor with tangent vector type of X, where both M and p are just help variables.