Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations

McGreivy, Nick; Hakim, Ammar

doi:10.1038/s42256-024-00897-5

Title: Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations

Journal Article · 25 September 2024 · Nature Machine Intelligence

DOI: https://doi.org/10.1038/s42256-024-00897-5 · OSTI ID:2475825

^[1]; Hakim, Ammar ^[2]

Princeton Univ., NJ (United States); Princeton Plasma Physics Laboratory (PPPL), Princeton, NJ (United States)
Princeton Plasma Physics Laboratory (PPPL), Princeton, NJ (United States)

One of the most promising applications of machine learning in computational physics is to accelerate the solution of partial differential equations (PDEs). The key objective of machine-learning-based PDE solvers is to output a sufficiently accurate solution faster than standard numerical methods, which are used as a baseline comparison. Here, we first perform a systematic review of the ML-for-PDE-solving literature. Out of all of the articles that report using ML to solve a fluid-related PDE and claim to outperform a standard numerical method, we determine that 79% (60/76) make a comparison with a weak baseline. Second, we find evidence that reporting biases are widespread, especially outcome reporting and publication biases. We conclude that ML-for-PDE-solving research is overoptimistic: weak baselines lead to overly positive results, while reporting biases lead to under-reporting of negative results. To a large extent, these issues seem to be caused by factors similar to those of past reproducibility crises: researcher degrees of freedom and a bias towards positive results. We call for bottom-up cultural changes to minimize biased reporting as well as top-down structural reforms to reduce perverse incentives for doing so.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Princeton Plasma Physics Laboratory (PPPL), Princeton, NJ (United States)

Sponsoring Organization:: USDOE Advanced Research Projects Agency - Energy (ARPA-E); USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR). Scientific Discovery through Advanced Computing (SciDAC)

Grant/Contract Number:: AC02-09CH11466

OSTI ID:: 2475825

Journal Information:: Nature Machine Intelligence, Journal Name: Nature Machine Intelligence Journal Issue: 10 Vol. 6; ISSN 2522-5839

Publisher:: Springer NatureCopyright Statement

Country of Publication:: United States

Language:: English

References (112)

Data-driven projection method in fluid simulation Yang, Cheng; Yang, Xubo; Xiao, Xiangyun Computer Animation and Virtual Worlds, Vol. 27, Issue 3-4 https://doi.org/10.1002/cav.1695	journal	May 2016
The Case Against Registered Reports Gundersen, Odd Erik AI Magazine, Vol. 42, Issue 1 https://doi.org/10.1002/j.2371-9621.2021.tb00016.x	journal	March 2021
hypre: A Library of High Performance Preconditioners Falgout, Robert D.; Yang, Ulrike Meier; Goos, Gerhard Computational Science — ICCS 2002: International Conference Amsterdam, The Netherlands, April 21–24, 2002 Proceedings, Part III https://doi.org/10.1007/3-540-47789-6_66	book	April 2002
Numerical Methods for Conservation Laws LeVeque, Randall J. Lectures in Mathematics. ETH Zürich https://doi.org/10.1007/978-3-0348-8629-1	book	January 1992
Enhancement of shock-capturing methods via machine learning Stevens, Ben; Colonius, Tim Theoretical and Computational Fluid Dynamics, Vol. 34, Issue 4 https://doi.org/10.1007/s00162-020-00531-1	journal	May 2020
Machine Learning in Drug Discovery: A Review Dara, Suresh; Dhamercherla, Swetha; Jadav, Surender Singh Artificial Intelligence Review, Vol. 55, Issue 3 https://doi.org/10.1007/s10462-021-10058-4	journal	August 2021
An Edge Detector Based on Artificial Neural Network with Application to Hybrid Compact-WENO Finite Difference Scheme Wen, Xiao; Don, Wai Sun; Gao, Zhen Journal of Scientific Computing, Vol. 83, Issue 3 https://doi.org/10.1007/s10915-020-01237-6	journal	June 2020
Improved Architectures and Training Algorithms for Deep Operator Networks Wang, Sifan; Wang, Hanwen; Perdikaris, Paris Journal of Scientific Computing, Vol. 92, Issue 2 https://doi.org/10.1007/s10915-022-01881-0	journal	June 2022
Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next Cuomo, Salvatore; Di Cola, Vincenzo Schiano; Giampaolo, Fabio Journal of Scientific Computing, Vol. 92, Issue 3 https://doi.org/10.1007/s10915-022-01939-z	journal	July 2022
Numerical Computation of Partial Differential Equations by Hidden-Layer Concatenated Extreme Learning Machine Ni, Naxian; Dong, Suchuan Journal of Scientific Computing, Vol. 95, Issue 2 https://doi.org/10.1007/s10915-023-02162-0	journal	March 2023
Publication bias in meta-analysis its causes and consequences Thornton, A. Journal of Clinical Epidemiology, Vol. 53, Issue 2 https://doi.org/10.1016/S0895-4356(99)00161-4	journal	February 2000
Graph neural network-accelerated Lagrangian fluid simulation Li, Zijie; Farimani, Amir Barati Computers & Graphics, Vol. 103 https://doi.org/10.1016/j.cag.2022.02.004	journal	April 2022
Neural Green’s function for Laplacian systems Tang, Jingwei; Azevedo, Vinicius C.; Cordonnier, Guillaume Computers & Graphics, Vol. 107 https://doi.org/10.1016/j.cag.2022.07.016	journal	October 2022
Local randomized neural networks with discontinuous Galerkin methods for partial differential equations Sun, Jingbo; Dong, Suchuan; Wang, Fei Journal of Computational and Applied Mathematics, Vol. 445 https://doi.org/10.1016/j.cam.2024.115830	journal	August 2024
Local extreme learning machines and domain decomposition for solving linear and nonlinear partial differential equations Dong, Suchuan; Li, Zongwei Computer Methods in Applied Mechanics and Engineering, Vol. 387 https://doi.org/10.1016/j.cma.2021.114129	journal	December 2021
Accelerating high order discontinuous Galerkin solvers using neural networks: 1D Burgers’ equation de Lara, Fernando Manrique; Ferrer, Esteban Computers & Fluids, Vol. 235 https://doi.org/10.1016/j.compfluid.2021.105274	journal	March 2022
Deep neural operators as accurate surrogates for shape optimization Shukla, Khemraj; Oommen, Vivek; Peyvan, Ahmad Engineering Applications of Artificial Intelligence, Vol. 129 https://doi.org/10.1016/j.engappai.2023.107615	journal	March 2024
Adaptive time stepping for fluid-structure interaction solvers Mayr, M.; Wall, W. A.; Gee, M. W. Finite Elements in Analysis and Design, Vol. 141 https://doi.org/10.1016/j.finel.2017.12.002	journal	March 2018
Residual-based physics-informed transfer learning: A hybrid method for accelerating long-term CFD simulations via deep learning Jeon, Joongoo; Lee, Juhyeong; Vinuesa, Ricardo International Journal of Heat and Mass Transfer, Vol. 220 https://doi.org/10.1016/j.ijheatmasstransfer.2023.124900	journal	March 2024
Negative results in computer vision: A perspective Borji, Ali Image and Vision Computing, Vol. 69 https://doi.org/10.1016/j.imavis.2017.10.001	journal	January 2018
Deep neural networks to correct sub-precision errors in CFD Haridas, Akash; Vadlamani, Nagabhushana Rao; Minamoto, Yuki Applications in Energy and Combustion Science, Vol. 12 https://doi.org/10.1016/j.jaecs.2022.100081	journal	December 2022
An artificial neural network as a troubled-cell indicator Ray, Deep; Hesthaven, Jan S. Journal of Computational Physics, Vol. 367 https://doi.org/10.1016/j.jcp.2018.04.029	journal	August 2018
Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations Raissi, M.; Perdikaris, P.; Karniadakis, G. E. Journal of Computational Physics, Vol. 378 https://doi.org/10.1016/j.jcp.2018.10.045	journal	February 2019
Detecting troubled-cells on two-dimensional unstructured grids using a neural network Ray, Deep; Hesthaven, Jan S. Journal of Computational Physics, Vol. 397 https://doi.org/10.1016/j.jcp.2019.07.043	journal	November 2019
Controlling oscillations in high-order Discontinuous Galerkin schemes using artificial viscosity tuned by neural networks Discacciati, Niccolò; Hesthaven, Jan S.; Ray, Deep Journal of Computational Physics, Vol. 409 https://doi.org/10.1016/j.jcp.2020.109304	journal	May 2020
Constraint-aware neural networks for Riemann problems Magiera, Jim; Ray, Deep; Hesthaven, Jan S. Journal of Computational Physics, Vol. 409 https://doi.org/10.1016/j.jcp.2020.109345	journal	May 2020
Controlling oscillations in spectral methods by local artificial viscosity governed by neural networks Schwander, Lukas; Ray, Deep; Hesthaven, Jan S. Journal of Computational Physics, Vol. 431 https://doi.org/10.1016/j.jcp.2021.110144	journal	April 2021
DeepM&Mnet: Inferring the electroconvection multiphysics fields based on operator approximation by neural networks Cai, Shengze; Wang, Zhicheng; Lu, Lu Journal of Computational Physics, Vol. 436 https://doi.org/10.1016/j.jcp.2021.110296	journal	July 2021
A data-driven physics-informed finite-volume scheme for nonclassical undercompressive shocks Bezgin, Deniz A.; Schmidt, Steffen J.; Adams, Nikolaus A. Journal of Computational Physics, Vol. 437 https://doi.org/10.1016/j.jcp.2021.110324	journal	July 2021
Beyond the Courant-Friedrichs-Lewy condition: Numerical methods for the wave problem using deep learning Ovadia, Oded; Kahana, Adar; Turkel, Eli Journal of Computational Physics, Vol. 442 https://doi.org/10.1016/j.jcp.2021.110493	journal	October 2021
A modified batch intrinsic plasticity method for pre-training the random coefficients of extreme learning machines Dong, Suchuan; Li, Zongwei Journal of Computational Physics, Vol. 445 https://doi.org/10.1016/j.jcp.2021.110585	journal	November 2021
DeepM&Mnet for hypersonics: Predicting the coupled flow and finite-rate chemistry behind a normal shock using neural-network approximation of operators Mao, Zhiping; Lu, Lu; Marxen, Olaf Journal of Computational Physics, Vol. 447 https://doi.org/10.1016/j.jcp.2021.110698	journal	December 2021
When and why PINNs fail to train: A neural tangent kernel perspective Wang, Sifan; Yu, Xinling; Perdikaris, Paris Journal of Computational Physics, Vol. 449 https://doi.org/10.1016/j.jcp.2021.110768	journal	January 2022
WENO3-NN: A maximum-order three-point data-driven weighted essentially non-oscillatory scheme Bezgin, Deniz A.; Schmidt, Steffen J.; Adams, Nikolaus A. Journal of Computational Physics, Vol. 452 https://doi.org/10.1016/j.jcp.2021.110920	journal	March 2022
On computing the hyperparameter of extreme learning machines: Algorithm and application to computational PDEs, and comparison with classical and high-order finite elements Dong, Suchuan; Yang, Jielin Journal of Computational Physics, Vol. 463 https://doi.org/10.1016/j.jcp.2022.111290	journal	August 2022
Neural operator prediction of linear instability waves in high-speed boundary layers Clark Di Leoni, Patricio; Lu, Lu; Meneveau, Charles Journal of Computational Physics, Vol. 474 https://doi.org/10.1016/j.jcp.2022.111793	journal	February 2023
Long-time integration of parametric evolution equations with physics-informed DeepONets Wang, Sifan; Perdikaris, Paris Journal of Computational Physics, Vol. 475 https://doi.org/10.1016/j.jcp.2022.111855	journal	February 2023
Accelerating high order discontinuous Galerkin solvers using neural networks: 3D compressible Navier-Stokes equations Manrique de Lara, Fernando; Ferrer, Esteban Journal of Computational Physics, Vol. 489 https://doi.org/10.1016/j.jcp.2023.112253	journal	September 2023
Predicting the propagation of acoustic waves using deep convolutional neural networks Alguacil, Antonio; Bauerheim, Michaël; Jacob, Marc C. Journal of Sound and Vibration, Vol. 512 https://doi.org/10.1016/j.jsv.2021.116285	journal	November 2021
Backpropagation algorithms and Reservoir Computing in Recurrent Neural Networks for the forecasting of complex spatiotemporal dynamics Vlachas, P. R.; Pathak, J.; Hunt, B. R. Neural Networks, Vol. 126 https://doi.org/10.1016/j.neunet.2020.02.016	journal	June 2020
LordNet: An efficient neural network for learning to solve parametric partial differential equations without simulated data Huang, Xinquan; Shi, Wenlei; Gao, Xiaotian Neural Networks, Vol. 176 https://doi.org/10.1016/j.neunet.2024.106354	journal	August 2024
Immersed boundary-physics informed machine learning approach for fluid–solid coupling Fang, Dehong; Tan, Jifu Ocean Engineering, Vol. 263 https://doi.org/10.1016/j.oceaneng.2022.112360	journal	November 2022
Leakage and the reproducibility crisis in machine-learning-based science Kapoor, Sayash; Narayanan, Arvind Patterns, Vol. 4, Issue 9 https://doi.org/10.1016/j.patter.2023.100804	journal	September 2023
Physics-informed neural networks for data-driven simulation: Advantages, limitations, and opportunities Fernández de la Mata, Félix; Gijón, Alfonso; Molina-Solana, Miguel Physica A: Statistical Mechanics and its Applications, Vol. 610 https://doi.org/10.1016/j.physa.2022.128415	journal	January 2023
A machine learning based solver for pressure Poisson equations Chen, Ruilin; Jin, Xiaowei; Li, Hui Theoretical and Applied Mechanics Letters, Vol. 12, Issue 5 https://doi.org/10.1016/j.taml.2022.100362	journal	September 2022
The cumulative effect of reporting and citation biases on the apparent efficacy of treatments: the case of depression de Vries, Y. A.; Roest, A. M.; de Jonge, P. Psychological Medicine, Vol. 48, Issue 15 https://doi.org/10.1017/S0033291718001873	journal	August 2018
The first AI simulation of a black hole Nemmen, Rodrigo; Duarte, Roberta; Navarro, João P. Proceedings of the International Astronomical Union, Vol. 15, Issue S359 https://doi.org/10.1017/S1743921320003981	journal	March 2019
Poisson CNN: Convolutional neural networks for the solution of the Poisson equation on a Cartesian mesh Özbay, Ali Girayhan; Hamzehloo, Arash; Laizet, Sylvain Data-Centric Engineering, Vol. 2 https://doi.org/10.1017/dce.2021.7	journal	January 2021
Performance and accuracy assessments of an incompressible fluid solver coupled with a deep convolutional neural network Ajuria Illarramendi, Ekhi; Bauerheim, Michaël; Cuenot, Bénédicte Data-Centric Engineering, Vol. 3 https://doi.org/10.1017/dce.2022.2	journal	January 2022
Numerical investigation of minimum drag profiles in laminar flow using deep learning surrogates Chen, Li-Wei; Cakal, Berkay A.; Hu, Xiangyu Journal of Fluid Mechanics, Vol. 919 https://doi.org/10.1017/jfm.2021.398	journal	June 2021
Learned turbulence modelling with differentiable fluid solvers: physics-based loss functions and optimisation horizons List, Björn; Chen, Li-Wei; Thuerey, Nils Journal of Fluid Mechanics, Vol. 949 https://doi.org/10.1017/jfm.2022.738	journal	September 2022
Deep Learning in Chemistry Mater, Adam C.; Coote, Michelle L. Journal of Chemical Information and Modeling, Vol. 59, Issue 6 https://doi.org/10.1021/acs.jcim.9b00266	journal	May 2019
Benchmarking of Machine Learning Ocean Subgrid Parameterizations in an Idealized Model Ross, Andrew; Li, Ziwei; Perezhogin, Pavel Journal of Advances in Modeling Earth Systems, Vol. 15, Issue 1 https://doi.org/10.1029/2022MS003258	journal	January 2023
Raise standards for preclinical cancer research Begley, C. Glenn; Ellis, Lee M. Nature, Vol. 483, Issue 7391 https://doi.org/10.1038/483531a	journal	March 2012
Metascience could rescue the ‘replication crisis’ Schooler, Jonathan W. Nature, Vol. 515, Issue 7525 https://doi.org/10.1038/515009a	journal	November 2014
Transcription initiation complex structures elucidate DNA opening Plaschka, C.; Hantsche, M.; Dienemann, C. Nature, Vol. 533, Issue 7603 https://doi.org/10.1038/nature17990	journal	May 2016
Believe it or not: how much can we rely on published data on potential drug targets? Prinz, Florian; Schlange, Thomas; Asadullah, Khusru Nature Reviews Drug Discovery, Vol. 10, Issue 9 https://doi.org/10.1038/nrd3439-c1	journal	August 2011
Best practices in machine learning for chemistry Artrith, Nongnuch; Butler, Keith T.; Coudert, François-Xavier Nature Chemistry, Vol. 13, Issue 6 https://doi.org/10.1038/s41557-021-00716-z	journal	May 2021
A manifesto for reproducible science Munafò, Marcus R.; Nosek, Brian A.; Bishop, Dorothy V. M. Nature Human Behaviour, Vol. 1, Issue 1 https://doi.org/10.1038/s41562-016-0021	journal	January 2017
Navigating the pitfalls of applying machine learning in genomics Whalen, Sean; Schreiber, Jacob; Noble, William S. Nature Reviews Genetics, Vol. 23, Issue 3 https://doi.org/10.1038/s41576-021-00434-9	journal	November 2021
Highly accurate protein structure prediction with AlphaFold Jumper, John; Evans, Richard; Pritzel, Alexander Nature https://doi.org/10.1038/s41586-021-03819-2	journal	July 2021
Physics-informed machine learning Karniadakis, George Em; Kevrekidis, Ioannis G.; Lu, Lu Nature Reviews Physics, Vol. 3, Issue 6 https://doi.org/10.1038/s42254-021-00314-5	journal	May 2021
Machine learning in the search for new fundamental physics Karagiorgi, Georgia; Kasieczka, Gregor; Kravitz, Scott Nature Reviews Physics, Vol. 4, Issue 6 https://doi.org/10.1038/s42254-022-00455-1	journal	May 2022
Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators Lu, Lu; Jin, Pengzhan; Pang, Guofei Nature Machine Intelligence, Vol. 3, Issue 3 https://doi.org/10.1038/s42256-021-00302-5	journal	March 2021
Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans Roberts, Michael; Driggs, Derek; Thorpe, Matthew Nature Machine Intelligence, Vol. 3, Issue 3 https://doi.org/10.1038/s42256-021-00307-0	journal	March 2021
Enhancing computational fluid dynamics with machine learning Vinuesa, Ricardo; Brunton, Steven L. Nature Computational Science, Vol. 2, Issue 6 https://doi.org/10.1038/s43588-022-00264-7	journal	June 2022
Promising directions of machine learning for partial differential equations Brunton, Steven L.; Kutz, J. Nathan Nature Computational Science, Vol. 4, Issue 7 https://doi.org/10.1038/s43588-024-00643-2	journal	June 2024
Machine Learning in Medicine Rajkomar, Alvin; Dean, Jeffrey; Kohane, Isaac New England Journal of Medicine, Vol. 380, Issue 14 https://doi.org/10.1056/NEJMra1814259	journal	April 2019
Teaching the incompressible Navier–Stokes equations to fast neural surrogate models in three dimensions Wandel, Nils; Weinmann, Michael; Klein, Reinhard Physics of Fluids, Vol. 33, Issue 4 https://doi.org/10.1063/5.0047428	journal	April 2021
Attention-enhanced neural network models for turbulence simulation Peng, Wenhui; Yuan, Zelong; Wang, Jianchun Physics of Fluids, Vol. 34, Issue 2 https://doi.org/10.1063/5.0079302	journal	February 2022
Linear attention coupled Fourier neural operator for simulation of three-dimensional turbulence Peng, Wenhui; Yuan, Zelong; Li, Zhijie Physics of Fluids, Vol. 35, Issue 1 https://doi.org/10.1063/5.0130334	journal	January 2023
Misrepresentation and distortion of research in biomedical literature Boutron, Isabelle; Ravaud, Philippe Proceedings of the National Academy of Sciences, Vol. 115, Issue 11 https://doi.org/10.1073/pnas.1710755115	journal	March 2018
Learning data-driven discretizations for partial differential equations Bar-Sinai, Yohai; Hoyer, Stephan; Hickey, Jason Proceedings of the National Academy of Sciences, Vol. 116, Issue 31 https://doi.org/10.1073/pnas.1814058116	journal	July 2019
Machine learning–accelerated computational fluid dynamics Kochkov, Dmitrii; Smith, Jamie A.; Alieva, Ayya Proceedings of the National Academy of Sciences, Vol. 118, Issue 21 https://doi.org/10.1073/pnas.2101784118	journal	May 2021
Can physics-informed neural networks beat the finite element method? Grossmann, Tamara G.; Komorowska, Urszula Julia; Latz, Jonas IMA Journal of Applied Mathematics, Vol. 89, Issue 1 https://doi.org/10.1093/imamat/hxae011	journal	May 2024
Black hole weather forecasting with deep learning: a pilot study Duarte, Roberta; Nemmen, Rodrigo; Navarro, João Paulo Monthly Notices of the Royal Astronomical Society, Vol. 512, Issue 4 https://doi.org/10.1093/mnras/stac665	journal	March 2022
Solving inverse problems in physics by optimizing a discrete loss: Fast and accurate learning without neural networks Karnakov, Petr; Litvinov, Sergey; Koumoutsakos, Petros PNAS Nexus, Vol. 3, Issue 1 https://doi.org/10.1093/pnasnexus/pgae005	journal	December 2023
The Systematic Review Aromataris, Edoardo; Pearson, Alan AJN, American Journal of Nursing, Vol. 114, Issue 3 https://doi.org/10.1097/01.NAJ.0000444496.24228.2c	journal	March 2014
The fundamental principles of reproducibility Gundersen, Odd Erik Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 379, Issue 2197 https://doi.org/10.1098/rsta.2020.0210	journal	March 2021
Learned discretizations for passive scalar advection in a two-dimensional turbulent flow Zhuang, Jiawei; Kochkov, Dmitrii; Bar-Sinai, Yohai Physical Review Fluids, Vol. 6, Issue 6 https://doi.org/10.1103/PhysRevFluids.6.064605	journal	June 2021
Machine learning and the physical sciences Carleo, Giuseppe; Cirac, Ignacio; Cranmer, Kyle Reviews of Modern Physics, Vol. 91, Issue 4 https://doi.org/10.1103/RevModPhys.91.045002	journal	December 2019
A Poisson’s Equation Solver Based on Neural Network Precondtioned CG Method Shao, Tianchen; Shan, Tao; Li, Maokun 2022 International Applied Computational Electromagnetics Society Symposium (ACES-China) https://doi.org/10.1109/ACES-China56081.2022.10064917	conference	December 2022
Study on a Fast Solver for Poisson’s Equation Based on Deep Learning Technique Shan, Tao; Tang, Wei; Dang, Xunwang IEEE Transactions on Antennas and Propagation, Vol. 68, Issue 9 https://doi.org/10.1109/TAP.2020.2985172	journal	September 2020
A Novel CNN-Based Poisson Solver for Fluid Simulation Xiao, Xiangyun; Zhou, Yanqing; Wang, Hui IEEE Transactions on Visualization and Computer Graphics, Vol. 26, Issue 3 https://doi.org/10.1109/TVCG.2018.2873375	journal	March 2020
Deep Fluids: A Generative Network for Parameterized Fluid Simulations Kim, Byungsoo; Azevedo, Vinicius C.; Thuerey, Nils Computer Graphics Forum, Vol. 38, Issue 2 https://doi.org/10.1111/cgf.13619	journal	May 2019
Nonreplicable publications are cited more than replicable ones Serra-Garcia, Marta; Gneezy, Uri Science Advances, Vol. 7, Issue 21 https://doi.org/10.1126/sciadv.abd1705	journal	May 2021
Learning the solution operator of parametric partial differential equations with physics-informed DeepONets Wang, Sifan; Wang, Hanwen; Perdikaris, Paris Science Advances, Vol. 7, Issue 40 https://doi.org/10.1126/sciadv.abi8605	journal	October 2021
Artificial intelligence faces reproducibility crisis Hutson, Matthew Science, Vol. 359, Issue 6377 https://doi.org/10.1126/science.359.6377.725	journal	February 2018
Estimating the reproducibility of psychological science Science, Vol. 349, Issue 6251 https://doi.org/10.1126/science.aac4716	journal	August 2015
Selective reporting bias of harm outcomes within studies: findings from a cohort of systematic reviews Saini, P.; Loke, Y. K.; Gamble, C. BMJ, Vol. 349, Issue nov21 3 https://doi.org/10.1136/bmj.g6501	journal	November 2014
Harms of outcome switching in reports of randomised trials: CONSORT perspective Altman, Douglas G.; Moher, David; Schulz, Kenneth F. BMJ https://doi.org/10.1136/bmj.j396	journal	February 2017
Data-dredging bias Erasmus, Adrian; Holman, Bennett; Ioannidis, John P. A. BMJ Evidence-Based Medicine, Vol. 27, Issue 4 https://doi.org/10.1136/bmjebm-2020-111584	journal	December 2021
Catalogue of bias: selective outcome reporting bias Thomas, Elizabeth T.; Heneghan, Carl BMJ Evidence-Based Medicine, Vol. 27, Issue 6 https://doi.org/10.1136/bmjebm-2021-111845	journal	February 2022
Multigrid-Augmented Deep Learning Preconditioners for the Helmholtz Equation Azulay, Yael; Treister, Eran SIAM Journal on Scientific Computing, Vol. 45, Issue 3 https://doi.org/10.1137/21M1433514	journal	August 2022
Strong Stability-Preserving High-Order Time Discretization Methods Gottlieb, Sigal; Shu, Chi-Wang; Tadmor, Eitan SIAM Review, Vol. 43, Issue 1 https://doi.org/10.1137/S003614450036757X	journal	January 2001
Improvements that don't add up Armstrong, Timothy G.; Moffat, Alistair; Webber, William Proceedings of the 18th ACM conference on Information and knowledge management https://doi.org/10.1145/1645953.1646031	conference	November 2009
REAL ML: Recognizing, Exploring, and Articulating Limitations of Machine Learning Research Smith, Jessie J.; Amershi, Saleema; Barocas, Solon 2022 ACM Conference on Fairness, Accountability, and Transparency https://doi.org/10.1145/3531146.3533122	conference	June 2022
Physics-Informed Neural Operator for Learning Partial Differential Equations Li, Zongyi; Zheng, Hongkai; Kovachki, Nikola ACM / IMS Journal of Data Science, Vol. 1, Issue 3 https://doi.org/10.1145/3648506	journal	May 2024
Turbulence Modeling in the Age of Data Duraisamy, Karthik; Iaccarino, Gianluca; Xiao, Heng Annual Review of Fluid Mechanics, Vol. 51, Issue 1 https://doi.org/10.1146/annurev-fluid-010518-040547	journal	January 2019
Machine Learning for Social Science: An Agnostic Approach Grimmer, Justin; Roberts, Margaret E.; Stewart, Brandon M. Annual Review of Political Science, Vol. 24, Issue 1 https://doi.org/10.1146/annurev-polisci-053119-015921	journal	May 2021
False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant Simmons, Joseph P.; Nelson, Leif D.; Simonsohn, Uri Psychological Science, Vol. 22, Issue 11 https://doi.org/10.1177/0956797611417632	journal	October 2011
The Extent and Consequences of P-Hacking in Science Head, Megan L.; Holman, Luke; Lanfear, Rob PLOS Biology, Vol. 13, Issue 3 https://doi.org/10.1371/journal.pbio.1002106	journal	March 2015
Why Most Published Research Findings Are False Ioannidis, John P. A. PLoS Medicine, Vol. 2, Issue 8 https://doi.org/10.1371/journal.pmed.0020124	journal	August 2005
“Positive” Results Increase Down the Hierarchy of the Sciences Fanelli, Daniele PLoS ONE, Vol. 5, Issue 4 https://doi.org/10.1371/journal.pone.0010068	journal	April 2010
Meaningless comparisons lead to false optimism in medical machine learning DeMasi, Orianna; Kording, Konrad; Recht, Benjamin PLOS ONE, Vol. 12, Issue 9 https://doi.org/10.1371/journal.pone.0184604	journal	September 2017
Data created during random sampling from and systematic review of ML-for-PDE solving research Hakim, Ammar; McGreivy, Nicholas OSF https://doi.org/10.17605/OSF.IO/GQ5B3	text	January 2023
Experience report of physics-informed neural networks in fluid simulations: pitfalls and frustration Chuang, Pi-Yueh; Barba, Lorena Proceedings of the Python in Science Conference https://doi.org/10.25080/majora-212e5952-005	conference	January 2022
Deep Learning Surrogate for the Temporal Propagation and Scattering of Acoustic Waves Alguacil, Antonio; Bauerheim, Michael; Jacob, Marc C. AIAA Journal, Vol. 60, Issue 10 https://doi.org/10.2514/1.J061495	journal	October 2022
Critical Investigation of Failure Modes in Physics-informed Neural Networks Basir, Shamsulhaq; Senocak, Inanc AIAA SCITECH 2022 Forum https://doi.org/10.2514/6.2022-2353	conference	January 2022
Degrees of Freedom in Planning, Running, Analyzing, and Reporting Psychological Studies: A Checklist to Avoid p-Hacking Wicherts, Jelte M.; Veldkamp, Coosje L. S.; Augusteijn, Hilde E. M. Frontiers in Psychology, Vol. 7 https://doi.org/10.3389/fpsyg.2016.01832	journal	November 2016
Acceleration of Boltzmann Collision Integral Calculation Using Machine Learning Holloway, Ian; Wood, Aihua; Alekseenko, Alexander Mathematics, Vol. 9, Issue 12 https://doi.org/10.3390/math9121384	journal	June 2021
A machine learning framework for data driven acceleration of computations of di erential equations Mishra, Siddhartha Mathematics in Engineering, Vol. 1, Issue 1 https://doi.org/10.3934/Mine.2018.1.118	journal	January 2018

Similar Records

Optical neural engine for solving scientific partial differential equations

Journal Article · 2025 · Nature Communications · OSTI ID:2566929

Tang, Yingheng; Chen, Ruiyang; Lou, Minhan; +5 more

Leveraging Multitime Hamilton–Jacobi PDEs for Certain Scientific Machine Learning Problems

Journal Article · 2024 · SIAM Journal on Scientific Computing · OSTI ID:2441157

Chen, Paula; Meng, Tingwei; Zou, Zongren; +2 more

Overview of leakage scenarios in supervised machine learning

Journal Article · 2025 · Journal of Big Data · OSTI ID:2570884

Sasse, L.; Nicolaisen-Sobesky, E.; Dukart, J.; +9 more

Related Subjects

97 MATHEMATICS AND COMPUTING
machine learning
metascience
partial differential equations
reproducibility crisis

Title: Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations

Citation Formats

References (112)

Similar Records

Related Subjects