Manopt.stochastic_gradient_descentFunction
stochastic_gradient_descent(M, grad_f, p=rand(M); kwargs...)
stochastic_gradient_descent!(M, msgo, p; kwargs...)

perform a stochastic gradient descent. This can be perfomed in-place of p.

Input

• M::AbstractManifold: a Riemannian manifold $\mathcal M$
• grad_f: a gradient function, that either returns a vector of the gradients or is a vector of gradient functions
• p: a point on the manifold $\mathcal M$

alternatively to the gradient you can provide an ManifoldStochasticGradientObjective msgo, then using the cost= keyword does not have any effect since if so, the cost is already within the objective.

Keyword arguments

• cost=missing: you can provide a cost function for example to track the function value
• direction=StochasticGradient([zerovector](@extrefManifoldsBase.zerovector-Tuple{AbstractManifold, Any})(M, p))
• evaluation=AllocatingEvaluation(): specify whether the functions that return an array, for example a point or a tangent vector, work by allocating its result (AllocatingEvaluation) or whether they modify their input argument to return the result therein (InplaceEvaluation). Since usually the first argument is the manifold, the modified argument is the second.
• evaluation_order=:Random: specify whether to use a randomly permuted sequence (:FixedRandom:, a per cycle permuted sequence (:Linear) or the default :Random one.
• order_type=:RandomOder: a type of ordering of gradient evaluations. Possible values are :RandomOrder, a :FixedPermutation, :LinearOrder
• stopping_criterion=StopAfterIteration(1000): a functor indicating that the stopping criterion is fulfilled
• stepsize=default_stepsize(M, StochasticGradientDescentState): a functor inheriting from Stepsize to determine a step size
• order=[1:n]: the initial permutation, where n is the number of gradients in gradF.
• retraction_method=default_retraction_method(M, typeof(p)): a retraction $\operatorname{retr}$ to use, see the section on retractions

All other keyword arguments are passed to decorate_state! for state decorators or decorate_objective! for objective decorators, respectively.

Output

The obtained approximate minimizer $p^*$. To obtain the whole final state of the solver, see get_solver_return for details, especially the return_state= keyword.

source
Manopt.stochastic_gradient_descent!Function
stochastic_gradient_descent(M, grad_f, p=rand(M); kwargs...)
stochastic_gradient_descent!(M, msgo, p; kwargs...)

perform a stochastic gradient descent. This can be perfomed in-place of p.

Input

• M::AbstractManifold: a Riemannian manifold $\mathcal M$
• grad_f: a gradient function, that either returns a vector of the gradients or is a vector of gradient functions
• p: a point on the manifold $\mathcal M$

alternatively to the gradient you can provide an ManifoldStochasticGradientObjective msgo, then using the cost= keyword does not have any effect since if so, the cost is already within the objective.

Keyword arguments

• cost=missing: you can provide a cost function for example to track the function value
• direction=StochasticGradient([zerovector](@extrefManifoldsBase.zerovector-Tuple{AbstractManifold, Any})(M, p))
• evaluation=AllocatingEvaluation(): specify whether the functions that return an array, for example a point or a tangent vector, work by allocating its result (AllocatingEvaluation) or whether they modify their input argument to return the result therein (InplaceEvaluation). Since usually the first argument is the manifold, the modified argument is the second.
• evaluation_order=:Random: specify whether to use a randomly permuted sequence (:FixedRandom:, a per cycle permuted sequence (:Linear) or the default :Random one.
• order_type=:RandomOder: a type of ordering of gradient evaluations. Possible values are :RandomOrder, a :FixedPermutation, :LinearOrder
• stopping_criterion=StopAfterIteration(1000): a functor indicating that the stopping criterion is fulfilled
• stepsize=default_stepsize(M, StochasticGradientDescentState): a functor inheriting from Stepsize to determine a step size
• order=[1:n]: the initial permutation, where n is the number of gradients in gradF.
• retraction_method=default_retraction_method(M, typeof(p)): a retraction $\operatorname{retr}$ to use, see the section on retractions

All other keyword arguments are passed to decorate_state! for state decorators or decorate_objective! for objective decorators, respectively.

Output

The obtained approximate minimizer $p^*$. To obtain the whole final state of the solver, see get_solver_return for details, especially the return_state= keyword.

source

## State

Manopt.StochasticGradientDescentStateType
StochasticGradientDescentState <: AbstractGradientDescentSolverState

Store the following fields for a default stochastic gradient descent algorithm, see also ManifoldStochasticGradientObjective and stochastic_gradient_descent.

Fields

• p::P: a point on the manifold $\mathcal M$storing the current iterate
• direction: a direction update to use
• stop::StoppingCriterion: a functor indicating that the stopping criterion is fulfilled
• stepsize::Stepsize: a functor inheriting from Stepsize to determine a step size
• evaluation_order: specify whether to use a randomly permuted sequence (:FixedRandom:), a per cycle permuted sequence (:Linear) or the default, a :Random sequence.
• order: stores the current permutation
• retraction_method::AbstractRetractionMethod: a retraction $\operatorname{retr}$ to use, see the section on retractions

Constructor

StochasticGradientDescentState(M::AbstractManifold; kwargs...)

Create a StochasticGradientDescentState with start point p.

Keyword arguments

source

Additionally, the options share a DirectionUpdateRule, so you can also apply MomentumGradient and AverageGradient here. The most inner one should always be.

Manopt.StochasticGradientFunction
StochasticGradient(; kwargs...)
StochasticGradient(M::AbstractManifold; kwargs...)

Keyword arguments

• initial_gradient=zero_vector(M, p): a tangent vector at the point $p$ on the manifold $\mathcal M$
• p=rand(M): a point on the manifold $\mathcal M$to specify the initial value
Info

This function generates a ManifoldDefaultsFactory for StochasticGradientRule. For default values, that depend on the manifold, this factory postpones the construction until the manifold from for example a corresponding AbstractManoptSolverState is available.

source

which internally uses

Manopt.StochasticGradientRuleType
StochasticGradientRule<: AbstractGradientGroupDirectionRule

Create a functor (problem, state k) -> (s,X) to evaluate the stochatsic gradient, that is chose a random index from the state and use the internal field for evaluation of the gradient in-place.

The default gradient processor, which just evaluates the (stochastic) gradient or a subset thereof.

Fields

• X::T: a tangent vector at the point $p$ on the manifold $\mathcal M$

Constructor

StochasticGradientRule(M::AbstractManifold; p=rand(M), X=zero_vector(M, p))

Initialize the stochastic gradient processor with tangent vector type of X, where both M and p are just help variables.

stochastic_gradient_descent, [StochasticGradient])@ref)
The stochastic_gradient_descent solver requires the following functions of a manifold to be available