[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: [amibroker] Re: What metrics do you use for comparing systems ?



PureBytes Links

Trading Reference Links

In the context of back testing a trading system, I'd agree that profitable outliers will impair its merit when the results are measured on a risk adjusted basis. In other words, an ounce of good luck if you will becomes a paradox. When measuring results on a risk-adjusted basis it's probably a valid approach to exclude outliers provided that additional risk wasn't incurred to obtain the profit. Analogous to winning something from a lottery when the loss of the cost of a ticket isn't relevant. I don't think that's real-world in terms of designing and developing a trading system.


From: amibroker@xxxxxxxxxxxxxxx [mailto:amibroker@xxxxxxxxxxxxxxx] On Behalf Of Fred
Sent: Wednesday, December 14, 2005 5:19 PM
To: amibroker@xxxxxxxxxxxxxxx
Subject: [amibroker] Re: What metrics do you use for comparing systems ?

Price ?! risk or Equity Curve risk ?

The problem with Sharpe is that by it punishes upside "anomolies", if
you can call them that on the equity curve.  I'll take
uncharacteristic upside movement in the equity curve every day.

--- In amibroker@xxxxxxxxxxxxxxx, "cwest" <cwest@xxxx> wrote:
>
> I don't know anything about "Mulvaney," but it seems that the
essence of his
> comments which you quoted is an effort to discredit the use of the
standard
> deviation to measure risk. Not including the inherent risk of
outliers,
> whether a trading system was designed to capture those trades or
not,
> introduces skewing. The Sortino ratio does just that--it assumes
that only
> downside risk is important.

> Given that a trading system is intended to short-sell (as well),
then it's
> necessary to consider all price risk. I'm open to any suggestions
that might
> be a better performance benchmark, but so far measuring returns on a
> risk-adjusted basis is unequivocally the consensus.

> Colin West
>
>   _____ 
>
> From: amibroker@xxxxxxxxxxxxxxx [mailto:amibroker@xxxxxxxxxxxxxxx]
On Behalf
> Of eric paradis
> Sent: Wednesday, December 14, 2005 2:19 PM
> To: amibroker@xxxxxxxxxxxxxxx
> Subject: Re: [amibroker] Re: What metrics do you use for comparing
systems ?
>
>
> You will absolutely not have a high sharpe ratio if
> you have a long-term trend following system in either
> equities or futures.
> Trend followers have made many statements as to why
> low sharpe ratios exist in funds that average 20-100%
> returns in any given year due to outlying trades.
>
> The low Sharpe Ratio is due to the outlying winners,
> and their effect on the Sharpe Ratio calculation. This
> quote, taken from trendfollowing.com, discusses the
> negative side of using a Sharpe Ratio to calculate
> risk versus return-
>
> ( Mulvaney also notes that conventional measures of
> risk-adjusted returns (i.e. Sharpe ratio) miss the
> boat:
>
> "Implicitly using the standard deviation assumes that
> the returns are normally distributed. But in
>
> fact our returns stream is very positively skewed, and
> highly asymmetrical. Our standard
>
> deviation is extremely high but this is because of the
> positive outliers. The standard deviation
>
> involves squaring the deviations from the mean and the
> outliers are what really push it up. So a
>
> very strong case can be made that CTAs' performance is
> severely penalized by the Sharpe
>
> ratio." )
>
> -Eric
>
> --- sebastiandanconia <sebastiandanconia@xxxx>
> wrote:
>
> > "...fwiw, very few mutual funds exceed 1.0 MSR :).
> > Very good hedge
> > managers obtain 2.0+ MSR...
> >
> > Interesting!  Thanks, Colin.
> >
> >
> > S.
> >
> >
> > --- In amibroker@xxxxxxxxxxxxxxx, "cwest"
> > <cwest@xxxx> wrote:
> > >
> > > My favorite subject/issue--performance
> > measurement. The most
> > preferred
> > > benchmark by which investment and/or trading
> > results are measured
> > is the
> > > Sharpe ratio. However, imo there's a valid
> > modification one should
> > make to
> > > this measure. The Sharpe ratio assumes the
> > risk-free rate is the
> > interest
> > > rate of 90 day Government paper. That's
> > unreasonable as there are
> > plenty of
> > > alternatives that aren't classified as junk
> > paper--270 day BBB+
> > Corporate
> > > notes, for example. Even GM short-term paper is
> > still pretty much
> > risk-free!
> > > 
> > > Therefore, a modified Sharpe ratio (MSR) would be:
> > annualized daily
> > average
> > > return less the Corporate short-term interest
> > rate, divided by the
> > standard
> > > deviation of the annualized daily average return,
> > is 'my' benchmark
> > for
> > > investment and/or trading. When calculations are
> > annualized short-
> > term or
> > > long-term isn't too relevant.
> > > 
> > > If your trading systems can't exceed 1.5 MSR--the
> > higher the number
> > the
> > > better the performance--it's back to the drawing
> > board. fwiw, very
> > few
> > > mutual funds exceed 1.0 MSR :). Very good hedge
> > managers obtain
> > 2.0+ MSR.
> > > 
> > > Colin West
> > >
> > >   _____ 
> > >
> > > From: amibroker@xxxxxxxxxxxxxxx
> > [mailto:amibroker@xxxxxxxxxxxxxxx]
> > On Behalf
> > > Of Erik Skyba
> > > Sent: Tuesday, December 13, 2005 4:52 PM
> > > To: amibroker@xxxxxxxxxxxxxxx
> > > Subject: Re: [amibroker] What metrics do you use
> > for comparing
> > systems ?
> > >
> > >
> > > upi is the most important personally, some metric
> > that measures
> > > semi-standard deviation.
> > >
> > > ----- Original Message -----
> > > From: "eric paradis" <thechemistrybetweenus@xxxx>
> > > To: <amibroker@xxxxxxxxxxxxxxx>
> > > Sent: Tuesday, December 13, 2005 4:06 PM
> > > Subject: Re: [amibroker] What metrics do you use
> > for comparing
> > systems ?
> > >
> > >
> > > > drawdown, annual return, time from drawdown to
> > new
> > > > high, # of trades, win/loss %, avg winner , avg
> > loser.
> > > > Should be enough there to come up with a good
> > idea
> > > > about what is a good system.
> > > >
> > > > Eric
> > > >
> > > > --- Condottiere <manset01@xxxx> wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I am relatively new to trading and I've been
> > going
> > > > > through a variety of
> > > > > sources about system comparison and robustness
> > (i.e
> > > > > Kaufman and so on).
> > > > > However, I'd be really interested in what
> > serious
> > > > > traders find useful in
> > > > > the real world.
> > > > > For instance, do you use standard metrics or
> > custom
> > > > > metrics ?
> > > > >
> > > > > Manu thanks for your thoughts and your
> > guidance.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
>
______________________________________________________________________
> > _____
> > > > >
> > > > > Appel audio GRATUIT partout dans le monde avec
> > le
> > > > > nouveau Yahoo! Messenger
> > > > > Téléchargez cette version sur
> > > > > http://fr.messenger.yahoo.com
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > __________________________________________________
> > > > Do You Yahoo!?
> > > > Tired of spam?  Yahoo! Mail has the best spam
> > protection around
> > > > http://mail.yahoo.com
> > > >
> > > >
> > > >
> > > > Please note that this group is for discussion
> > between users only.
> > > >
> > > > To get support from AmiBroker please send an
> > e-mail directly to
> > > > SUPPORT {at} amibroker.com
> > > >
> > > > For other support material please check also:
> > > > http://www.amibroker.com/support.html
> > > >
> > > >
> > > > Yahoo! Groups Links
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > > Please note that this group is for discussion
> > between users only.
> > >
> > > To get support from AmiBroker please send an
> > e-mail directly to
> > > SUPPORT {at} amibroker.com
> > >
> > > For other support material please check also:
> > > http://www.amibroker.com/support.html
> > >
> > >
> > >
> > >
> > >
> > >   _____ 
> > >
> > > YAHOO! GROUPS LINKS
> > >
> > >
> > >      
> > > *      Visit your group "amibroker
> > > <http://groups.yahoo.com/group/amibroker> " on the
> > web.
> > >  
> > >
> > > *      To unsubscribe from this group, send an email
> > to:
> > >  amibroker-unsubscribe@xxxxxxxxxxxxxxx
> > >
> >
> <mailto:amibroker-unsubscribe@xxxxxxxxxxxxxxx?subject=Unsubscribe>
> >
> > >  
> >
> === message truncated ===
>
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
>
>
> Please note that this group is for discussion between users only.
>
> To get support from AmiBroker please send an e-mail directly to
> SUPPORT {at} amibroker.com
>
> For other support material please check also:
> http://www.amibroker.com/support.html
>
>
>
>
>
>   _____ 
>
> YAHOO! GROUPS LINKS
>
>
>      
> *      Visit your group "amibroker
> <http://groups.yahoo.com/group/amibroker> " on the web.
>  
>
> *      To unsubscribe from this group, send an email to:
>  amibroker-unsubscribe@xxxxxxxxxxxxxxx
> <mailto:amibroker-unsubscribe@xxxxxxxxxxxxxxx?subject=Unsubscribe>
>  
>
> *      Your use of Yahoo! Groups is subject to the Yahoo! Terms of
Service
> <http://docs.yahoo.com/info/terms/> .
>
>
>   _____
>






Please note that this group is for discussion between users only.

To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com

For other support material please check also:
http://www.amibroker.com/support.html





YAHOO! GROUPS LINKS