[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Optimization and Curve Fitting



PureBytes Links

Trading Reference Links

--- In amibroker@xxxx, "Tomasz Janeczko" <amibroker@xxxx> wrote:
> This brings the question of curve-fitting brought by David.
> The system is very sensitive to the changes of X1, Dbuy, Dsell 
> variables.
> The problem is that once we optimize for single stock 
> we get "perfect"
> parameters but only for this one stock and the history we just 
> used to
> optimize. What is needed is "out-of-sample" testing to 
> verify the results.
> ===============

I am far from an expert on optimization/curve fitting, but a few 
approaches have been helpful to me. 

1. Does it make sense LOGICALLY? I know from experience what trouble 
I can get into using logic (this one "should" go up) without 
confirmating the method with extensive backtesting. But I think there 
is a place for logic. If optimization gives a great result for an 
unusal variable, especially if the next nearest variables give poor 
results, I have no confidence in the results unless I can find a 
logical reason to explain why neighbouring variables could give such 
different results.

2. How consistently are the optimized results? One way to do this is 
to test the optimized values on "out of range" data (ie, data not 
used in the optimization runs). If the results are not similar my 
confidence is low. Another way I sometimes confirm optimization 
results is to "segment" the time period. I take my optimized values 
from say a 10 year period and try them on several 1 year periods 
within the 10 years. If an "optimized" value is going to give some 
years of negative results, I want to know ahead of time before 
putting real money in.

3. I sometimes choose the "safest" value even if it is not 
the "best". Consider the following:
Value, gain
1, 10%
2, 15%
3, 18%
4, 20%
5, 8%
6, 5%
Here I would perfer to use value 3 even though slightly better. My 
reasoning? If the market patterns shift slightly, 4 could easily 
become like 5. The shift would have to be greater to affect the 
returns from 3. 

4. I am always on the look out for out of pattern results: Consider
1, 5%
2, 10%
3, 20%
4, 5%
5, -2%
6, 30%
7, -3%
8, -5%
Here, I would be very suspicious of the optimised value of 6. I 
suspect it is due more to a random data set than to a repeating 
market event.

b