Approximate gradient projection method and backpropagation algorithm

Conference · Sat Dec 31 04:00:00 EST 1994

OSTI ID:36245

We analyze the convergence of an approximate gradient projection method for minimizing the sum of continuously differentiable functions over a nonempty closed convex set. In this method, the functions are aggregated and, at each iteration, a succession of gradient steps, one for each of the aggregate functions, is applied and the result is projected onto the convex set. We show that if the gradients of the functions are bounded and Lipschitz continuous over a certain level set and the stepsizes are chosen to be proportional to a certain residual squared or to be square summable, then every cluster point of the iterates is a stationary point. We apply these results to the backpropagation algorithm to obtain new deterministic convergence results for this algorithm. We will also report some numerical simulation results.

• OSTI ID: 36245
• Report Number(s): CONF-9408161--
• Country of Publication: United States
• Language: English

Related Subjects

99 GENERAL AND MISCELLANEOUS
ACCURACY
ALGORITHMS
CONVERGENCE
MATRICES
NEWTON METHOD
NUMERICAL SOLUTION
OPTIMIZATION
VARIATIONAL METHODS