Accurate numerical partials with applications to optimization

摘要

In this paper we show how to obtain O(h4) accurate approximations of the gradient and Hessian of all functions f : Rn → R, f ∈ C6(B(r)), B(r) an open ball of large radius r centered at the origin. Here h is the finite difference quotient increment. This O(h4) accuracy is attained by exploiting the classical numerical analytical notions of central difference quotients and extrapolation-to-the limit. The computational cost is 2n(n + 1) + 1 function evaluations per numerical gradient/Hessian. We give three numerical gradient/Hessian test case results. Also discussed is the performance of a prototype minimization algorithm using these accurate partials applied to two test cases: the Rosenbrock Banana function and a statistical parameter estimation maximum likelihood problem.