VARIABLE METRIC METHOD

Variable metric method - Originally referred to the Davidon-Fletcher-Powell (DFP) method , this is a family of methods that choose the direction vector in unconstrained optimization by the subproblem: d* in argmax(grad_f(x)d: ||d||=1), where ||d|| is the vector norm (or metric) defined by the quadratic form , d'Hd. With H symmetric and positive definite , the constraint d'Hd = 1 restricts d by being on a "circle" -- points that are "equidistant" from a stationary point, called the "center" (the origin in this case). By varying H, as in the DFP update, to capture the curvature of the objective function, f, we have a family of ascent algorithms. Besides DFP, if one chooses H=I, we have Cauchy's steepest ascent . If f is concave and one chooses H equal to the negative of the inverse hessian , we have the modified Newton's method .