Neural network approaches for parameterized optimal control

Journal Article · Sat Mar 01 00:00:00 EST 2025 · Foundations of Data Science

DOI:https://doi.org/10.3934/fods.2024042· OSTI ID:2481183

Verma, Deepanshu ^[1]; Winovich, Nick ^[2]; Ruthotto, Lars ^[3]; van Bloemen Waanders, Bart ^[2]

---

1. Clemson Univ., SC (United States)
2. Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)
3. Emory Univ., Atlanta, GA (United States)

Here, we consider numerical approaches for deterministic, finite-dimensional optimal control problems whose dynamics depend on unknown or uncertain parameters. We seek to amortize the solution over a set of relevant parameters in an offline stage to enable rapid decision-making and be able to react to changes in the parameter in the online stage. To tackle the curse of dimensionality arising when the state and/or parameter are high-dimensional, we represent the policy using neural networks. We compare two training paradigms: First, our model-based approach leverages the dynamics and definition of the objective function to learn the value function of the parameterized optimal control problem and obtain the policy using a feedback form. Second, we use actor-critic reinforcement learning to approximate the policy in a data-driven way. Using an example involving a two-dimensional convection-diffusion equation, which features high-dimensional state and parameter spaces, we investigate the accuracy and efficiency of both training paradigms. While both paradigms lead to a reasonable approximation of the policy, the model-based approach is more accurate and considerably reduces the number of PDE solves.

• Research Organization: Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)
• Sponsoring Organization: National Science Foundation (NSF); US Air Force Office of Scientific Research (AFOSR); USDOE National Nuclear Security Administration (NNSA); USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
• Grant/Contract Number: NA0003525
• OSTI ID: 2481183
• Report Number(s): SAND--2024-16667J
• Journal Information: Foundations of Data Science, Journal Name: Foundations of Data Science Journal Issue: 1 Vol. 7; ISSN 2639-8001
• Publisher: AIMS
• Country of Publication: United States
• Language: English

Related Subjects

97 MATHEMATICS AND COMPUTING
Hamilton-Jacobi-Bellman equation
PDE constrained optimization
Reinforcement Learning
actor-critic
high dimensional optimal control
neural networks
policy optimization