|
PureBytes Links
Trading Reference Links
|
Jim:
Dennis Miller is correct in noting the amount of work involved.
Large.
However, even if you put the work in, what are you testing?
Two different systems, not optimization efficacy.
Why?
System one:
Two linear regression values, each optimized over
5,000 bars and trading signals taken on cross over / under.
Simple.
System one carries an optimization load of 2
(2 degrees of freedom) for every 5,000 bars.
System two:
Same logic, but divided into 1,000 bar segments, the rolling
window size. Therefore, you have 2 degrees of freedom for
every window. The parameter to sample size load is 5 times
greater.
System two MIGHT perform better or it might perform worse
than system one depending on how well the 1,000 bar window
matches the resonance frequency of the underlying data set.
Conclusion: the proposed method for rolling window
optimization does NOT discriminate between the efficacy of
one kind of optimization versus another, but discriminates
between a 1st order polynomial curve fit versus a higher
order polynomial curve fit system.
A serious methodological error.
For further discussion, see Tuchar Chande: Beyond Technical
Analysis, Second Edition; Wiley & Sons, 2001; pages 44 - 48.
Regards,
Leslie
Jim Johnson wrote:
>
> Hello Leslie,
>
> As Dennis Miller says, "I could be wrong but..." it would be a lot of
> work but it seems you could test your walk forward method directly
> rather than rely on assumptions. As an example:
>
> optimize a system on 1990 data.
>
> check its results on 1991 data.
>
> optimize on 1991.
>
> check its results on 1992.
>
> etc etc.
>
> The results from "checking" would give you the actual performance of
> the "system"--system now defined as trading rules plus
> re-optimization.
>
> Best regards,
> Jim Johnson mailto:jejohn@xxxxxxxxxxx
>
> --
> Thursday, October 31, 2002, 10:56:33 PM, you wrote:
>
> LW> Mark:
> LW> Perhaps if you rethink your question more in terms of 'what is
> LW> the hypothesis that I am attempting to test', you might find
> LW> more enlightenment than if you were searching for
> LW> "producing more reliable results".
>
> LW> The theory behind walk forward optimization is that the market
> LW> inefficiency you plan to trade in the near future has existed
> LW> only for limited periods in the past and may manifest themselves
> LW> for only a little while longer. Certain markets, say SP day
> LW> trade SEEM to display this characteristic.
>
> LW> If this theory were true in a strict sense then rolling window
> LW> would give you the best results assuming that (a) you selected
> LW> the in sample size correctly; and (b) you selected the out of
> LW> sample size correctly. This might a a bit hard to do :-)
>
> LW> A weaker version of this theory is that DUH something happened
> LW> that changed the market and the 'old' market efficiency was
> LW> replaced by the 'new' market inefficiency that you are going to
> LW> get rich on -- assuming you find the magic DUH moment
> LW> to begin your in sample optimization at the right time.
>
> LW> Notice that both the strong and the weak form of the walk forward
> LW> optimization hypothesis contain untestable auxiliary hypothesis
> LW> about beginning periods, in sample, and out of sample size.
>
> LW> So, it this a 'good theory'?
> LW> A philospher of science would have serious reservations about it.
> LW> A philosopher would want to know how your system perform
> LW> using 'regular' optimization versus 'walk forward optimization'
> LW> using TRUE OUT OF SAMPLE DATA.
> LW> For the philosopher the 'goodness of a theory' is relative to
> LW> the goodness of other theories.
> LW> If you cannot even determine how two theories compare to each
> LW> other, a philosopher might advise you to abandon your useless
> LW> theory, (or write a book about it to collect royalties).
>
> LW> Enjoy,
> LW> Leslie
>
> LW> mark.keenan@xxxxxxxxxxxxxx wrote:
> >>
> >> Any opinions or experience on whether anchored forward optimization
> >> (keeping the start date fixed) produces more reliable results than the more
> >> common walk forward optimization, where the "in sample window is rolled
> >> forward"
> >>
> >> Most walk forward examples in books describe optimizing over 2 years for
> >> example - trading the optimized parameters over the next six months - and
> >> the rolling the whole two year window forward by six months and the
> >> repeating.
> >>
> >> My own testing has found anchored to be much more effective - I guess
> >> building on the concept that if your going to optimize anyway then use as
> >> much data as possible.
> >>
> >> I have been doing some work on a Double Linear Regression slope system on
> >> index futures where I optimize over the whole data range (3 years - as
> >> before three years the market was traded on the floor - is now screen
> >> traded for the last 3 years) trade the result on the next month and then
> >> add this recent "traded" month to the data series - re-optimize, trade on
> >> the following month etc. Therefore as time passes my in sample data length
> >> keeps getting bigger and bigger.
> >>
> >> On OUT OF SAMPLE trading - WITHOUT any stops yet - still doing the MAE
> >> analysis the system is showing the following results using anchored
> >> analysis.
> >>
> >> Total Trades 143
> >> % Winning 51.05%
> >>
> >> Average Losing 2.95%
> >> Average Winning 3.35%
> >> Average Trade 0.31%
> >>
> >> Largest Winner 18.63%
> >> Largest Loser 10.6% (would obviously be stopped out as below)
> >>
> >> Profit Factor 1.27
> >> MAE average 2.31% (will be re-running results with a 2.50% stop
> >> I think)
> >>
> >> Any views on the above??
> >>
> >> MK
> >>
> >> (sorry about disclaimer)
> >>
> >> ---------------------------------------------------------------------------
> >> This message (including any attachments) is confidential and may be
> >> privileged. If you have received it by mistake please notify the sender by
> >> return e-mail and delete this message from your system. Any unauthorised
> >> use or dissemination of this message in whole or in part is strictly
> >> prohibited. Please note that e-mails are susceptible to change.
> >> ABN AMRO Bank N.V. (including its group companies) shall not be liable for
> >> the improper or incomplete transmission of the information contained in
> >> this communication nor for any delay in its receipt or damage to your
> >> system. ABN AMRO Bank N.V. (or its group companies) does not guarantee that
> >> the integrity of this communication has been maintained nor that this
> >> communication is free of viruses, interceptions or interference.
> >> ---------------------------------------------------------------------------
--
Regards,
Leslie Walko
610-688-2442
--
"Life is a tragedy for those who feel, a comedy for those who
think"
Horace Walpole, 4th earl of Orford, in a letter dated about 1770
|