skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

This content will become publicly available on April 5, 2021

Title: Reinforcement learning for bluff body active flow control in experiments and simulations

Abstract

We have demonstrated the effectiveness of reinforcement learning (RL) in bluff body flow control problems both in experiments and simulations by automatically discovering active control strategies for drag reduction in turbulent flow. Specifically, we aimed to maximize the power gain efficiency by properly selecting the rotational speed of two small cylinders, located parallel to and downstream of the main cylinder. By properly defining rewards and designing noise reduction techniques, and after an automatic sequence of tens of towing experiments, the RL agent was shown to discover a control strategy that is comparable to the optimal strategy found through lengthy systematically planned control experiments. Subsequently, these results were verified by simulations that enabled us to gain insight into the physical mechanisms of the drag reduction process. While RL has been used effectively previously in idealized computer flow simulation studies, this study demonstrates its effectiveness in experimental fluid mechanics and verifies it by simulations, potentially paving the way for efficient exploration of additional active flow control strategies in other complex fluid mechanics applications.

Authors:
ORCiD logo; ; ORCiD logo; ORCiD logo; ORCiD logo
Publication Date:
Sponsoring Org.:
USDOE
OSTI Identifier:
1670468
Grant/Contract Number:  
SC0019453
Resource Type:
Published Article
Journal Name:
Proceedings of the National Academy of Sciences of the United States of America
Additional Journal Information:
Journal Name: Proceedings of the National Academy of Sciences of the United States of America Journal Volume: 117 Journal Issue: 42; Journal ID: ISSN 0027-8424
Publisher:
Proceedings of the National Academy of Sciences
Country of Publication:
United States
Language:
English

Citation Formats

Fan, Dixia, Yang, Liu, Wang, Zhicheng, Triantafyllou, Michael S., and Karniadakis, George Em. Reinforcement learning for bluff body active flow control in experiments and simulations. United States: N. p., 2020. Web. doi:10.1073/pnas.2004939117.
Fan, Dixia, Yang, Liu, Wang, Zhicheng, Triantafyllou, Michael S., & Karniadakis, George Em. Reinforcement learning for bluff body active flow control in experiments and simulations. United States. doi:10.1073/pnas.2004939117.
Fan, Dixia, Yang, Liu, Wang, Zhicheng, Triantafyllou, Michael S., and Karniadakis, George Em. Mon . "Reinforcement learning for bluff body active flow control in experiments and simulations". United States. doi:10.1073/pnas.2004939117.
@article{osti_1670468,
title = {Reinforcement learning for bluff body active flow control in experiments and simulations},
author = {Fan, Dixia and Yang, Liu and Wang, Zhicheng and Triantafyllou, Michael S. and Karniadakis, George Em},
abstractNote = {We have demonstrated the effectiveness of reinforcement learning (RL) in bluff body flow control problems both in experiments and simulations by automatically discovering active control strategies for drag reduction in turbulent flow. Specifically, we aimed to maximize the power gain efficiency by properly selecting the rotational speed of two small cylinders, located parallel to and downstream of the main cylinder. By properly defining rewards and designing noise reduction techniques, and after an automatic sequence of tens of towing experiments, the RL agent was shown to discover a control strategy that is comparable to the optimal strategy found through lengthy systematically planned control experiments. Subsequently, these results were verified by simulations that enabled us to gain insight into the physical mechanisms of the drag reduction process. While RL has been used effectively previously in idealized computer flow simulation studies, this study demonstrates its effectiveness in experimental fluid mechanics and verifies it by simulations, potentially paving the way for efficient exploration of additional active flow control strategies in other complex fluid mechanics applications.},
doi = {10.1073/pnas.2004939117},
journal = {Proceedings of the National Academy of Sciences of the United States of America},
number = 42,
volume = 117,
place = {United States},
year = {2020},
month = {10}
}

Journal Article:
Free Publicly Available Full Text
This content will become publicly available on April 5, 2021
Publisher's Version of Record

Save / Share:

Works referenced in this record:

Wakes in Stratified Fluids
journal, January 1979


Vortex Shedding from Oscillating Bluff Bodies
journal, January 1984


Flow control with rotating cylinders
journal, July 2017

  • Schulmeister, James C.; Dahl, J. M.; Weymouth, G. D.
  • Journal of Fluid Mechanics, Vol. 825
  • DOI: 10.1017/jfm.2017.395

Efficient collective swimming by harnessing vortices through deep reinforcement learning
journal, May 2018

  • Verma, Siddhartha; Novati, Guido; Koumoutsakos, Petros
  • Proceedings of the National Academy of Sciences, Vol. 115, Issue 23
  • DOI: 10.1073/pnas.1800923115

Controlled gliding and perching through deep-reinforcement-learning
journal, September 2019


Closed-Loop Turbulence Control: Progress and Challenges
journal, August 2015

  • Brunton, Steven L.; Noack, Bernd R.
  • Applied Mechanics Reviews, Vol. 67, Issue 5
  • DOI: 10.1115/1.4031175

Wakes Behind Blunt Bodies
journal, January 1990


Control of Flow Over a Bluff Body
journal, January 2008


Reinforcement Learning and Wavelet Adapted Vortex Methods for Simulations of Self-propelled Swimmers
journal, January 2014

  • Gazzola, Mattia; Hejazialhosseini, Babak; Koumoutsakos, Petros
  • SIAM Journal on Scientific Computing, Vol. 36, Issue 3
  • DOI: 10.1137/130943078

Machine Learning for Fluid Mechanics
journal, September 2019


An entropy-viscosity large eddy simulation study of turbulent flow in a flexible pipe
journal, November 2018

  • Wang, Zhicheng; Triantafyllou, Michael S.; Constantinides, Yiannis
  • Journal of Fluid Mechanics, Vol. 859
  • DOI: 10.1017/jfm.2018.808

Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control
journal, February 2019

  • Rabault, Jean; Kuchta, Miroslav; Jensen, Atle
  • Journal of Fluid Mechanics, Vol. 865
  • DOI: 10.1017/jfm.2019.62

Mastering the game of Go without human knowledge
journal, October 2017

  • Silver, David; Schrittwieser, Julian; Simonyan, Karen
  • Nature, Vol. 550, Issue 7676
  • DOI: 10.1038/nature24270

A Linear Systems Approach to Flow Control
journal, January 2007


Simultaneous CFD evaluation of VIV suppression using smaller control cylinders
journal, August 2015


Flow Navigation by Smart Microswimmers via Reinforcement Learning
journal, April 2017


Fluid directed rigid body control using deep reinforcement learning
journal, August 2018

  • Ma, Pingchuan; Tian, Yunsheng; Pan, Zherong
  • ACM Transactions on Graphics, Vol. 37, Issue 4
  • DOI: 10.1145/3197517.3201334

Learning to soar in turbulent environments
journal, August 2016

  • Reddy, Gautam; Celani, Antonio; Sejnowski, Terrence J.
  • Proceedings of the National Academy of Sciences, Vol. 113, Issue 33
  • DOI: 10.1073/pnas.1606075113

A robotic Intelligent Towing Tank for learning complex fluid-structure dynamics
journal, November 2019


Mapping the properties of the vortex-induced vibrations of flexible cylinders in uniform oncoming flow
journal, October 2019

  • Fan, Dixia; Wang, Zhicheng; Triantafyllou, Michael S.
  • Journal of Fluid Mechanics, Vol. 881
  • DOI: 10.1017/jfm.2019.738

Drag reduction of a bluff body using adaptive control methods
journal, August 2006

  • Beaudoin, Jean-François; Cadot, Olivier; Aider, Jean-Luc
  • Physics of Fluids, Vol. 18, Issue 8
  • DOI: 10.1063/1.2236305

On the formation and suppression of vortex ‘shedding’ at low Reynolds numbers
journal, September 1990


Periodic Flow Phenomena
journal, January 1972


Fabrication and characterization of folded foils supporting streamwise traveling waves
journal, November 2019


Vortex Dynamics in the Cylinder Wake
journal, January 1996