Masters Testing and Tuning Market Trading Systems

Algorithms in C++
1. ed
ISBN: 978-1-4842-4173-8
Verlag: Apress
Format: PDF
Kopierschutz: 1 - PDF Watermark

E-Book, Englisch, 325 Seiten

Build, test, and tune financial, insurance or other market trading systems using C++ algorithms and statistics. You've had an idea and have done some preliminary experiments, and it looks promising. Where do you go from here?  Well, this book discusses and dissects this case study approach.   Seemingly good backtest performance isn't enough to justify trading real money. You need to perform rigorous statistical tests of the system's validity. Then, if basic tests confirm the quality of your idea, you need to tune your system, not just for best performance, but also for robust behavior in the face of inevitable market changes. Next, you need to quantify its expected future behavior, assessing how bad its real-life performance might actually be, and whether you can live with that. Finally, you need to find its theoretical performance limits so you know if its actual trades conform to this theoretical expectation, enabling you to dump the system if it does not live up to expectations.This book does not contain any sure-fire, guaranteed-riches trading systems. Those are a dime a dozen... But if you have a trading system, this book will provide you with a set of tools that will help you evaluate the potential value of your system, tweak it to improve its profitability, and monitor its on-going performance to detect deterioration before it fails catastrophically. Any serious market trader would do well to employ the methods described in this book.What You Will LearnSee how the 'spaghetti-on-the-wall' approach to trading system development can be done legitimatelyDetect overfitting early in developmentEstimate the probability that your system's backtest results could have been due to just good luckRegularize a predictive model so it automatically selects an optimal subset of indicator candidatesRapidly find the global optimum for any type of parameterized trading systemAssess the ruggedness of your trading system against market changesEnhance the stationarity and information content of your proprietary indicatorsNest one layer of walkforward analysis inside another layer to account for selection bias in complex trading systemsCompute a lower bound on your system's mean future performanceBound expected periodic returns to detect on-going system deterioration before it becomes severeEstimate the probability of catastrophic drawdown  Who This Book Is For Experienced C++ programmers, developers, and software engineers.  Prior experience with rigorous statistical procedures to evaluate and maximize the quality of systems is recommended as well.  


Timothy Masters received a PhD in mathematical statistics with a specialization in numerical computing. Since then he has continuously worked as an independent consultant for government and industry. His early research involved automated feature detection in high-altitude photographs while he developed applications for flood and drought prediction, detection of hidden missile silos, and identification of threatening military vehicles. Later he worked with medical researchers in the development of computer algorithms for distinguishing between benign and malignant cells in needle biopsies. For the last twenty years he has focused primarily on methods for evaluating automated financial market trading systems. He has authored five books on practical applications of predictive modeling: Practical Neural Network Recipes in C++ (Academic Press, 1993); Signal and Image Processing with Neural Networks (Wiley, 1994); Advanced Algorithms for Neural Networks (Wiley, 1995); Neural, Novel, and Hybrid Algorithms for Time Series Prediction (Wiley, 1995); Data Mining Algorithms in C++ (Apress, 2018); Assessing and Improving Prediction and Classification (Apress, 2018); Deep Belief Nets in C++ and CUDA C: Volume 1 (Apress, 2018); and Deep Belief Nets in C++ and CUDA C: Volume 2 (Apress, 2018).

Autoren/Hrsg.

Masters, Timothy

Inhaltsverzeichnis

1;Table of Contents;4
2;About the Author;8
3;About the Technical Reviewer;9
4;Chapter 1: Introduction;10
4.1;The Target Audience, and Overview of Contents;10
4.1.1;What’s in This Book;10
4.1.2;What’s Not in This Book;12
4.2;About Trading Systems;13
4.2.1;Market Prices and Returns;14
4.2.2;Two Types of Automated Trading Systems;15
4.2.3;The Agony of Believing the Computer;16
4.2.4;Future Leak Is More Dangerous Than You May Think;16
4.2.5;The Percent Wins Fallacy;17
5;Chapter 2: Pre-optimization Issues;19
5.1;Assessing and Improving Stationarity;19
5.1.1;The STATN Program;21
5.1.2;Improving Location Stationarity by Oscillating;25
5.1.3;Extreme Stationarity Induction;27
5.2;Measuring Indicator Information with Entropy;28
5.2.1;Computing the Relative Entropy of an Indicator;30
5.2.2;Entropy Impacts Predictive Model Quality;32
5.2.3;Improving the Entropy of an Indicator;33
5.2.4;Monotonic Tail-Only Cleaning;37
5.2.4.1;The ENTROPY Program;39
6;Chapter 3: Optimization Issues;42
6.1;Regularizing a Linear Model;42
6.1.1;Overview of the Regularized Model;43
6.1.2;Beta Adjustment with Guaranteed Convergence;47
6.1.3;Differential Case Weighting;48
6.1.4;Rapid Computation with Covariance Updates;49
6.1.4.1;Preparatory Code;50
6.1.5;Outline of the Beta Optimization Process;53
6.1.6;Code for Beta Optimization;55
6.1.7;Descending a Lambda Path;62
6.1.8;Optimizing Lambda with Cross Validation;66
6.1.9;The CD_MA Program;70
6.2;Making a Linear Model Nonlinear;74
6.3;Differential Evolution: A Universal Nonlinear Optimizer;76
6.3.1;The DIFF_EV.CPP Routine for Differential Evolution;82
7;Chapter 4: Post-optimization Issues;97
7.1;Cheap Bias Estimates;97
7.1.1;The StocBias Class;98
7.2;Cheap Parameter Relationships;102
7.3;Parameter Sensitivity Curves;114
7.3.1;Putting It All Together Trading OEX;118
7.3.1.1;The Trading System;118
7.3.1.2;Linking Criterion Routines;121
7.3.1.3;Application to Trading OEX;123
8;Chapter 5: Estimating Future Performance I: Unbiased Trade Simulation;127
8.1;In-Sample and Out-of-Sample Performance;127
8.1.1;The TrnBias Program to Demonstrate Training Bias;129
8.1.2;Selection Bias;130
8.1.2.1;Interlude: What Does Unbiased Really Mean?;131
8.1.2.2;Selection Bias, Continued;132
8.1.2.3;The SelBias Program;133
8.1.3;Walkforward Analysis;135
8.1.4;Future Leak by Unobvious IS/OOS Overlap;137
8.1.4.1;Error Variance Inflation with Multiple-Bar Lookaheads;138
8.1.4.2;The General Walkforward Algorithm;140
8.1.4.3;C++ Code for the Algorithm;141
8.1.4.4;Date-Dependent Walkforward;143
8.1.4.5;Exploring Walkforward Blunders;143
8.1.4.6;Testing Robustness Against Nonstationarity;148
8.1.5;Cross-Validation Analysis;149
8.1.5.1;Unobvious IS/OOS Overlap;149
8.1.5.2;The Fully General Cross-Validation Algorithm;150
8.1.5.3;C++ Code for the General Algorithm;152
8.1.5.4;Cross Validation Can Have Pessimistic Bias;156
8.1.5.5;Cross Validation Can Have Optimistic Bias;156
8.1.5.6;Cross Validation Does Not Reflect Real Life;156
8.1.6;Special Precautions for Algorithmic Trading;157
8.1.6.1;Converting Unknown-Lookahead Systems to Single-Bar;161
8.1.6.2;Unbounded Lookback Can Subtly Happen;162
8.2;Comparing Cross Validation with Walkforward: XVW;162
8.2.1;Computationally Symmetric Cross Validation;164
8.2.1.1;The CSCV Algorithm: Intuition and General Statement;166
8.2.2;What Does This Test Actually Measure?;169
8.2.2.1;C++ Code for the CSCV Superiority Test;170
8.2.2.2;An Example with SPX;175
8.3;Nested Walkforward Analysis;178
8.3.1;The Nested Walkforward Algorithm;180
8.3.2;A Practical Application of Nested Walkforward;185
8.4;An Example Using S&P 100 Components;193
8.5;Cross Validation Nested Inside Walkforward;194
9;Chapter 6: Estimating Future Performance II: Trade Analysis;198
9.1;Handling Dynamic Trading Systems;198
9.1.1;Unknown Lookahead to Single Bars, Revisited;199
9.2;Profit per Bar? Per Trade? Per Time?;200
9.2.1;Analyzing Completed Trade Returns Is Problematic;201
9.2.2;The PER_WHAT Program;203
9.3;A Lower Bound for Mean Future Returns;214
9.3.1;Brief Digression: Hypothesis Tests;215
9.3.2;So, How Do We Use This Probability?;217
9.3.3;Parametric P-Values;221
9.3.4;Parametric Confidence Intervals;223
9.3.5;Lower Confidence Bounds and Hypothesis Tests;227
9.4;Bootstrap Confidence Intervals;227
9.4.1;The Pivot and Percentile Methods;228
9.4.2;The BCa Bootstrap Algorithm;230
9.4.3;The BOOT_CONF.CPP Subroutines;232
9.4.4;The BOUND_MEAN Program and Results with SPX;237
9.4.5;Beware of Bootstrapping Ratios;243
9.5;Bounding Future Returns;246
9.5.1;Deriving a Lower Bound from Empirical Quantiles;247
9.5.2;Confidence in the Computed Lower Bound;249
9.5.3;What About an Upper Bound on Future Returns?;252
9.5.4;The CONFTEST Program: Overview;253
9.5.5;The CONFTEST Program: Code;256
9.5.6;The BND_RET Program;262
9.6;Bounding Drawdown;267
9.6.1;Intuition Gone Wrong;268
9.6.2;Bootstrapping Drawdown Bounds;270
9.6.3;The DRAWDOWN Program;272
9.6.4;Experiments with the DRAWDOWN Program;282
9.6.5;The CHOOSER_DD Program;284
10;Chapter 7: Permutation Tests;287
10.1;Overview of Permutation Testing;287
10.2;Testing a Fully Specified Trading System;289
10.2.1;Testing the Training Process;290
10.2.2;Walkforward Testing a Trading System Factory;291
10.2.3;Permutation Testing of Predictive Models;293
10.3;The Permutation Testing Algorithm;295
10.3.1;Extending the Algorithm for Selection Bias;296
10.4;Partitioning Total Return of a Trading System;298
10.4.1;Essential Permutation Algorithms and Code;302
10.4.1.1;Simple Permutation;302
10.4.2;Permuting Simple Market Prices;303
10.4.3;Permuting Multiple Markets with an Offset;305
10.4.3.1;Permuting Price Bars;309
10.5;Example: P-Value and Partitioning;314
10.5.1;Example: Training with Next Bar Returns;316
10.6;Example: Permuting Multiple Markets;320
11;Index;323