The biggest mistake you can make in background testing

Spark Global Limited Reports:

Reverse testing tests trading strategies against historical data.

It provides traders and analysts with a research tool to test their current strategies, find new ones, or figure out which factors contribute most to their success.

Most background testing is done in code, using libraries from languages like Python and R, so if you have coding skills, you can background test almost anything.

There are also no-code or low-code post-test solutions, such as TradeStation or MultiCharts, but they are not flexible enough for professionals.

An example of backtracking would be to download ten years of market data, say 2007 to 2017, and test the trading strategies you’re interested in. You need to be able to turn your strategy into a strictly quantitative format, such as “buy when the 50-day moving average crosses the 200-day moving average” rather than “buy the bull flag.”

After the computer runs the test, you get a stock curve, a list of trades, and indicators like the Sharpe ratio and maximum fall.

Here is an example of an analysis “tear page” generated when running a backtest on the Zipline Trader library (an updated version of Quantopian’s Now defunct Zipline library) :
This tear page is, of course, customized by the user for illustrative purposes.

Backtracking can feel like magic.

Download a bunch of market data, run some parameters, tweak them slightly until the test shows a smooth stock curve, and pretty soon, you have a profitable trading system that can print money. The only obstacle seems to be learning to code and understanding which metrics and parameters will yield the best results.

But this is simple data mining, and using this kind of backtracking method is not useful because tests have no predictive power. It represents a mistake that even the most seasoned researchers make over and over again: mistaking correlation for causation. I’ll use an example to illustrate this:

You’ve run thousands of tests using machine learning to find the most profitable trading system.

The best tests show that the best strategy is to buy XYZ shares at 10:53 a.m. Tuesday, after the stock had risen at least 1.04% the previous day with an RSI of 38 or more. Obviously, this makes no sense. If I told you this was my trading system, you’d laugh at me for having such a pointless system that doesn’t take advantage of any real market imbalances.

All this test does is find the perfect parameters to match accurate historical data.

While data mining is rarely so pointless when it comes to backtesting, you must constantly check every deviation you introduce into your testing.

Here are some of the most common mistakes that are crucial if you don’t want to burn money when trading through tested strategies.

©Spark Global Limited Financial information & The content of the website comes from the Internet, and any infringement links will be deleted.