Gradient Descent

General implementation of gradient descent algorithm.

More of a learning exercise for myself.

mr_utils.optimization.gd.gd(f, grad, x0, alpha=None, maxiter=1000000.0, tol=1e-08)

Gradient descent algorithm.

Parameters:

• f (callable) – Function to be optimized.
• grad (callable) – Function that computes the gradient of f.
• x0 (array_like) – Initial point to start to start descent.
• alpha (callable or float, optional) – Either a fixed step size or a function that returns step size.
• maxiter (int, optional) – Do not exceed this number of iterations.
• tol (float, optional) – Run until change in norm of gradient is within this number.

Returns:

• cur_x (array_like) – Estimate of optimal choice of x.
• int – Number of iterations.