Advanced Walk Forward Backtesting Method

Effective backtesting is not trivial and a science in itself, here is a brief introduction of the two main methods:

Method1: Common unrealistic Long term curve fitting Method

Usually you apply your trading strategy to a long period (blue zone - called In-Sample-Test) and then optimize the result,
by running through many variations of several input parameter combinations in your strategy (Permutations).
Out of all these permutations you then select the best result (Max equity, min draw down etc)
and then start live trading with this optimized set of strategy parameter (red zone - called Out-Of-Sample Test)
Below is a visual representation of this process:

As you might have experienced yourself - the likelihood of this optimized (usually phantastic) result to repeat in the future is very slim,
and disappointing actual trading results usually occur, because just you have "curve-fitted" the strategy input parameter to that particular long period.
Optimizing your strategy on this overall maximum equity, is usually used by various Genetic Optimizers.

Method2: Realistic Walk forward Strategy Backtesting and Optimization of Filter/Sort/Period Settings

In the advanced "walk-forward" method, you optimize on many shorter periods (In-Sample) and then
verify repeatedly in the "future" (Out-Of-Sample) if those strategy parameter still produce favorable results
Below is a visual representation of the walk-forward-backtesting and optimization (WFO) process:

In this example only the first 4 weeks (blue block - called the "In Sample" period) were used in the back testing.
After all the many permutations of the strategy input parameters have been calculated, the WFO filter is applied;
which eliminates many loosing or unrealistic results outright. Then the WFO sort  is applied - yielding one final permutation;
which is then applied in the following 5th week (green block - called the "Out of Sample" period).
This process is repeated to all the available data (in this example 12 weeks, with 8 WFO optimization runs);
each time the WFO filter and sort will select a different input parameter permutation for the out-of-sample week.
Finally all separate out-of-sample results are joined and merged into the final overall result.
The likelihood of repeating these results in the future in the "Out of Sample" period is very high and
before you risk your capital our WFO tools provide you with a realistic evaluation how your strategy would perform in the real live market.
Furthermore you can test many different WFO filter and sort algorithms to find the most effective WFO selection method.
This Walk Forward method has been used in the Grail Walk Forward Optimizer (GWFO) (Grail is no longer available),
but does allow you to customize the filter and sort methods, and daily or weekly periods and access the results on a exported spreadheet!

Walk Forward Backtesting Process
Start the Walk Forward Process by clicking on on the toolbar button (or from the menu Actions/Run WFO or Ctrl+R);
Here you can define the walk forward filter (how to eliminate permutations) and the sort criteria (how to pick the best):

To see the filter click on the "FilterMgr" button, shich will show the currently selected WF filter: Formula=($PF >= PFMin) AND ($PF <= PFMax) AND ($MaxGainCnt > MaxGainCntMin) AND ($MaxLossCnt < MaxLossCntMax)  PFMin=1.0 to 1.2 step 0.2 PFMax=1.8 to 2.2 step 0.2 MaxGainCntMin=6,5 MaxLossCntMax=6,5 You can also see that you can vary any number of the filter formula, i.e PFMin defines the minimum ProfitFactor ($PF) required and we are testing 1.0 and 1.2 (PFMin=1.0 to 1.2 step 0.2). In this example 24 total filter variations will be tested (2xPFMin,3xPFMax,2xMaxGain,2xMaxLoss). The "Sort Mgr" allows you to define the various sort criteria to be tested, in our example SortMix1 contain 4 different sort criteria: Sort1=$MaxTrdLoss/5 DESC,$Equity,$MaxDDAmt Sort2=$Equity,$MaxDDAmt,$MaxTrdLoss DESC Sort3=$PF,$Equity,$MaxTrdLoss DESC Sort4=$PFAvg,$Equity,$MaxTrdLoss DESC The "Period Mgr" allows you to select and even customize the in-sample and out-sample periods. In this example we run 4 weeks insample and test 1 week out of sample. To see the actual walk forward periods being testing, click on the "View Periods" button. The "Seq Mgr" allows you build clusters of different period definitions. Example: Yu can create cluster with the following 4 period definitions: 4Week/1Week, 5Week/1Week,6Week/1Week,7Week/1Week BWFMgr will the test each one and then you can compare the performance directly! Turn OFF all "Additional Diagnostic Options" to achive maximum processing speed. Click on START to initiate the walk forward process! When it completes BTWFMgr will display a message and add the results to the treeview (blue squares) BTWFMgr can calculate in a few short moments, what would be a manual extensive and error-prone effort. The progress bars will indicate the progress of the "Walk Forward Process". If you need the CPU power on your computer you can Pause and continue the processing later.

When the WF process has completed, a new branch with all the 96 WF results for each test run, is generate. Just right click on any WF result and select the requested function: usually the Out-of-Sample Chart  -or- WF Result Overview - LIST (as shown below) or click here (ISA=In-Sample, OSA=Out-of-Sample, WF=Walk Forward)

Period Manager

The Walk Forward "Period Mgr" allows you to select and even customize the in-sample and out-sample periods.
In this example above we run 4 weeks InSample and test 1 week out of sample.
To see the actual walk forward periods being testing, click on the "View Periods" button.
You can fin tune the periods using the following parameters in the "Walk Forward Optimization" config section

WFODayShift	Number of Days to shift from the calculated default dates: negative=backwards shift positive=forward shift Zero=no additional date shift
WFOBack	Run Walk Forward period creation from the back/last date (default from front/first date)
WFOWeekMonday	TRUE: For Weekly Periods - Adjusts the dates to always start on a Monday
WFOAddPeriod	Number of periods to append without historical data\nDefault = one(1) period for the first future period
OSAStyle	Define how Walk Forward Out Of Sample Positions are handled which cross/overlap across the begin or end of the OSA Period: 0 = include all overlapped positions in OSA Period 1 = Exclude ending/last overlapped position from OSA period 2 = Exclude the starting/first overlapped position from OSA period 3 = Exclude the starting/first and ending/last overlapped position from OSA period
OSAStyleVerbose	Switch OSA Overlap Check verbose mode ON/OFF
MaxWFOOpt	Maximum number of Walk Forward Optimizations to show directly in the Treeview  (Default=10) The remaining results are shown in the Other results branch
NormWFEqu	Normalize the WalkFwd Equity to the full period (default NO) Example: 50 Weeks of data using 10week ISA and 5week OSA would amplify by a factor of 25% (50/40)
WFAVerbose	Walk Forward diagnostic verbose level (0=none, 1=write diags to log)
WFOMaxOSAPerm	Number of parallel OutOfSample top permutations used (1=only top result, 5=Use 5 Top results and accumulate positions)
WFOMaxOSAPos	Number of parallel OutOfSample position in same bar (0=unlimited)
WFODiagPeriod	Specify the Walk Forward Period - to show detailed WFOMaxOSAPos calculations in Log File (0=no diagnostic) Example Diagnostic File PosDiag.txt  Processing Period#1 Processing Period: ISA 20131231 -> 20140127 Processing WalkFwdOpt#1 ... Checking Pos#00006 (Ranges= 4, Bar 474 -> 550): Vol=271 Short, Entry=20140131 10:30:00(36.8000), Exit=20140205 10:00:00(36.1300), PL=173.57$, Perm=45 Detected 4 Overlap(s) (Max=3)  ... Deleting 13 overlapping positions Processing WalkFwdOpt#2 Processing WalkFwdOpt#3 Processing WalkFwdOpt#4
SortPosByTime	Sort WalkFwd OSA Positions by Time Only (YES) - otherwise original Sequence (NO) You can also change the OSA Position sorting via the new right click Menu choices:
FindBarNbr	This switch in the "Initial Data Conversion" section - which allows you to force a FRESH search to find the Bar Number for each position (usually provided by the backtesting data!)

Displaying Walk Forward Results in 3D

Select Tools/Walk Forwared Results+Cluster Analysis.
Then select for example "Net Profit+Loss" and click on "3D View":


Click on the X/Y selection you would like to view (Example Period Matrix):


Now the three dimensional view appears -
were you can easily identify the various peaks with the best performance:

Another example is the 3D View of the "BTWFMgr Robustness":
The BTWFMgr Robustness is the calculated as the ratio between InSample and OutSample Profit/Loss,
but adjusted for the InSample/OutSample length - because the InSample period is usually much longer than the OutSample period:
A Robustness of 1.00 indicates a perfect match between the projected InSample result and the actual OutSample result, 
a robustness of 0.35(35%) indicates that the OutSample reached 35% of the extpected InSample Profit:

The same walk forward result matrix can also be viewed as a spreadsheet.
Just click on "Matrix" instead of the "3D View" and
then same view is exported and displayed as a spreadsheet:
(as a CSV file in the same folder with e*.btwf4 Walk Fwd data file)

Running a Sequence of Walk Forward Configurations

You can save and reload any WalkFwd configuration (Filter, Sort and Period definition) and even
define and run a sequence of configurations!

1.	Select the Filter, Sort and Period settings you would like to save as a WalkFwd configuration
2.	Click on "Save Config" button and enter your new name (Cfg1, etc)
3.	You can reload any of the saved WalkFwd configurations, just click on "Load Config" and select the requested Configuration.
4.	To define a WalkFwd Sequence: a) Click on "Sequence" and select "Create new Sequence" b) Enter the new name (Seq1 etc) c) In the new window, just list the configurations, separated by a comma     (Cfg1,Cfg2 etc)
5.	To run a WalkFwd sequence: a) Click on the "Sequence" button b) Click on Sequence you would like to run (Run Seq: SeqXXX) c) Click on "Start" d) Confirm starting the new WalkWd Sequence: e) The WalkFwd optimization will start and show you     the current Sequence and COnfiguration being calculated:
6.	Modify WalkFwd Sequence: a) Click on the "Sequence" button b) Click on "Modify Sequence" c) Select the sequence to modify d) Edit the Sequence in the new Notepad edit window and save/exit
7.	Delete a WalkFwd Sequence: a) Click on the "Sequence" button b) Click on "Delete Sequence" c) Select the sequence to modify d) Confirm the deletion
8.	File syntax: WalkFwd Configurations are stored in the C:/BTWFMgr base folder as WF{Cfg}.txt: Example: Filter=PFCheck Sort=SortMix2 Sequence=Daily WalkFwd Sequences are stored in the C:/BTWFMgr base folder as WS{Cfg}.txt Example: Cfg1,Cfg2,

Parallel OutOfSample(OSA) Permutations

To activate the parallel OutOfSample(OSA) function - adjust the "WFOMaxOSAPerm" parameter 
to the number of parallel OutOfSample top permutations used - Example: 5


Open the Walk Forward Optimization Box and start your optimization.
You will notice a new message - showing the parallel OSA permutations in the confirmation box:


When you view the result - via the "Show Walk Forward Result Overview - LIST",
you see the new "5 Best Permutations" column - which shows all the top 5 permutations from the filtered and sorted Walk Forward results:


When you view the actual OSA trades - via the "Show Out-of-Sample Positions - LIST",
you see the trades from MULTIPLE Permutations: 266+2903+2945,  Note: Some permutations do not have any trades in the Out-of-Sample periods (340, 344)!