skip to main content

Title: Simple Example of Backtest Overfitting (SEBO)

In the field of mathematical finance, a "backtest" is the usage of historical market data to assess the performance of a proposed trading strategy. It is a relatively simple matter for a present-day computer system to explore thousands, millions or even billions of variations of a proposed strategy, and pick the best performing variant as the "optimal" strategy "in sample" (i.e., on the input dataset). Unfortunately, such an "optimal" strategy often performs very poorly "out of sample" (i.e. on another dataset), because the parameters of the invest strategy have been oversit to the in-sample data, a situation known as "backtest overfitting". While the mathematics of backtest overfitting has been examined in several recent theoretical studies, here we pursue a more tangible analysis of this problem, in the form of an online simulator tool. Given a input random walk time series, the tool develops an "optimal" variant of a simple strategy by exhaustively exploring all integer parameter values among a handful of parameters. That "optimal" strategy is overfit, since by definition a random walk is unpredictable. Then the tool tests the resulting "optimal" strategy on a second random walk time series. In most runs using our online tool, the "optimal" strategymore » derived from the first time series performs poorly on the second time series, demonstrating how hard it is not to overfit a backtest. We offer this online tool, "Simple Example of Backtest Overfitting (SEBO)", to facilitate further research in this area.« less
Publication Date:
OSTI Identifier:
Report Number(s):
SEBO; 003275WKSTN00
R&D Project: 830404000; 2015-035
DOE Contract Number:
Resource Type:
Software Revision:
Software Package Number:
Software Package Contents:
Open Source Software package available from Lawrence Berkeley National Laboratory at the following URL:
Software CPU:
Open Source:
Source Code Available:
Research Org:
Lawrence Berkeley National Laboratory
Sponsoring Org:
Contributing Orgs:
K. John Wu, Stephanie Ger, Alex Sim, David H. Bailey, Marcos Lopez de Prado
Country of Publication:
United States

To initiate an order for this software, request consultation services, or receive further information, fill out the request form below. You may also reach us by email at: .

OSTI staff will begin to process an order for scientific and technical software once the payment and signed site license agreement are received. If the forms are not in order, OSTI will contact you. No further action will be taken until all required information and/or payment is received. Orders are usually processed within three to five business days.

Software Request