For a textual description, generate a backtest report and then
hover your mouse over any metric name. The tooltip will provide a
description of the metric. For most, a mathematical explanation can be
found via google.
Mike
--- In amibroker@xxxxxxxxxps.com,
"Potato Soup" <potatosoupz@...> wrote:
>
> Yes, I still don't have an explanation for what car is. And all
the others I listed.
>
> -----Original Message-----
> From: Keith McCombs <kmccombs@xx.>
> Date: Mon, 30 Nov 2009 14:05:23
> To: <amibroker@xxxxxxxxxps.com>
> Subject: Re: [amibroker] What's a good k ratio,
> and thoughts on when k ratio clashes with MDD and Sharpe?
>
> One thing that I would find most helpful, as far as performance
metrics
> are concerned, is a precise definition of each metric in
"AmiBroker
> User's Guide" (page 156 for v5.20), including AFL formulas to
produce
> results equivalent to those produced by AB back tester analysis.
>
> -- Keith
>
> Howard B wrote:
> >
> >
> > Hi PS --
> >
> > One way to get a feeling for values for metrics and objective
> > functions is run an optimization, giving you a range of
results. If
> > necessary, peek into the future so you are certain to get
some really
> > good results.
> >
> > Look through the list of results, pick some individual
results with a
> > variety of values for the metrics you want to learn about.
> >
> > Set the default value of the optimized variables to the
specific
> > values you chose.
> >
> > Run a single backtest. Plot the equity curve.
> >
> > When you have done a few of these and have a feeling for the
> > characteristics of the systems you would like to trade, print
out the
> > plot of the equity curves and write the values of the metrics
and
> > objective functions on the printout.
> >
> > -----------------------
> >
> > Your question about interpretation.
> >
> > My preference is for objective functions that reward equity
growth and
> > penalize drawdown. CAR/MDD, RAR/MDD, RRR, Recovery Factor,
K-Ratio,
> > Ulcer Performance Index, and Sharpe Ratio all do that. In all
of
> > these cases, larger values are better.
> >
> > The important results are the out-of-sample results. We are
looking
> > for logic and parameter values that not only perform well
in-sample,
> > but that also perform well out-of-sample.
> >
> > When you run walk forward tests, you will find that some
objective
> > functions give high rank to alternatives that do tend to
perform well
> > out-of-sample, while other objective functions select
alternatives
> > that often do not perform well out-of-sample. You will need
to run
> > some of your own tests to get a feeling for how these work on
your
> > trading systems.
> >
> > My experience is that using net profit is usually a poor
objective
> > function, although it is the default (and often the only)
selection
> > for some trading systems development platforms other than
AmiBroker.
> >
> > Fund managers are evaluated on the Sharpe Ratio of their
performance.
> > My experience is that systems selected using Sharpe Ratio
tend to
> > perform poorly out-of-sample.
> >
> > Standard Error is a measurement of the smoothness of the
equity line
> > -- smaller values are better. But optimizing to minimize
standard
> > error alone may give high ranks to systems that have trading
> > characteristics that you do not want. For example, if you
have the
> > options set so that the equity earns interest when ever it is
not in a
> > position, then using standard error will reward alternatives
that stay
> > in cash and trade infrequently.
> >
> > ------------------------
> >
> > One of the very valuable features of AmiBroker is the
capability for
> > the system developer to create whatever objective function he
or she
> > wants to use. It does not have to be limited to those that
are
> > distributed with AmiBroker and appear in the list of metrics.
You
> > might want to combine metrics -- for example to reward
alternatives
> > whose trading frequency suits your preferences, while also
rewarding
> > equity growth and penalizing drawdown.
> >
> > Thanks,
> > Howard
> >
> >
> >
> >
> > On Sat, Nov 28, 2009 at 11:33 PM, potatosoupz <potatosoupz@...
> > <mailto:potatosoupz@...>> wrote:
> >
> >
> >
> > I don't see any good definitions for the metrics below. I am
> > familiar of course with Sharpe, and Ulcer Index. I'm finding
it a
> > bit hard to reconcile the differences. How would you
reconcile a
> > backtest that has a max draw down % that is smaller than it's
> > profit %, a low ulcer index, a high Sharpe, but a very low k
ratio
> > (< .05)?
> >
> > Ulcer Index in my mind is one of the best metrics outside of a
> > much deeper quantitative treatment of things. Thoughts?
> >
> > CAR/MDD ?
> > RAR/MDD ?
> > Payoff Ratio ?
> > Standard Error ?
> > RRR ?
> > Recovery Factor ?
> >
> >
> >
>