|
PureBytes Links
Trading Reference Links
|
David, please post what you find in the differences on points only test.
----- Original Message -----
From: "equisdvc" <david_cardner@xxxxxxxxxxx>
To: <equismetastock@xxxxxxxxxxxxxxx>
Sent: Wednesday, December 03, 2003 4:02 PM
Subject: [EquisMetaStock Group] Re: Bugs in v8.01
> Tom,
> Thanks for taking the time to respond. I agree with everything you
> had to say with one exception. I have just run a points only
> comparison test and it did not match. Sent the results to Equis. I
> had all of the settings you recommended. It took about three months
> to find them after I bought v8.01.
> David
>
> --- In equismetastock@xxxxxxxxxxxxxxx, "Tom Sprunger"
> <tlsprunger@xxxx> wrote:
> > equisdvc, here is a summary of system test issues in 8.01 and how
> to get
> > around them I have posted a few times. Hope it helps you.
> >
> > -------------------
> > I spent a week or three figuring out how the MS v8 System Tester
> works. I
> > had problems getting the results from 8.01 to match 7.2. I had
> strange
> > signals, missing trades, multiple trades off one signal, etc. Had
> several
> > emails, phone conversations, etc with Equis. Bottom line is that
> they went
> > overboard in trying to make it realistic, to the point it is
> unrealistic
> > (my opinion). Especially if you are trying to develop and test
> systems.
> > However, there are two workarounds you can use to get valid results.
> >
> > So I won't forget what I learned I, wrote it up. Below tells how
> the tester
> > currently works and explains why the results give missing trades,
> multiple
> > trades, doesn't use your full equity, etc. and therefore provides
> invalid
> > results.
> >
> > Read below how 8.01 works, then look at the two methods to get
> valid results
> > if you are trying to develop and test systems.
> >
> > Equis has told me they are looking into changing it, but no commit,
> and no
> > timeframe for making the decision.
> > If you agree with me, send or call them and tell them to change
> it. At the
> > bottom, I list what I have recommended they do.
> >
> > --------------------------------------------------------------------
> --------
> >
> > The Metastock v8.x system tester works very differently than
> previous
> > versions. On the up side it can test multiple securities
> simultaneously.
> > However, its operation has differences that are very significant to
> a system
> > developer, and in reality are not anywhere near what would happen
> in the
> > real world.
> >
> > It is important to understand these differences because the v8
> tester can
> > give very different results from the v7 tester when using the same
> system on
> > the same security with the same time periods. In addition, because
> of the
> > v8 "features" you can get different results comparing the several
> systems on
> > the same security from the v7 tester. This also implies
> Optimization
> > results could be invalid.
> >
> > Later we will see how to get around these issues and limitations.
> >
> > Here's the major differences.
> >
> > The v8 tester has several new options that are key. The key
> differences
> > result from a "Broker" module which places orders.
> >
> > a. "Position Limit" This is accessible from the "General" screen
> on each
> > individual system test description. You will need to be very
> careful what
> > you put in here as you will see later. In addition, if you are
> comparing
> > different systems, you need to be sure that they all have the same
> number of
> > positions allowed or you may get unfair comparison results.
> >
> > b. Trading size. In the v7 tester the system traded 100% of your
> equity on
> > each trade. There was no option to do otherwise (except
> for "points only"
> > test). The v8 tester allows you to trade a constant number of
> shares, total
> > transaction cost, or a percentage of equity available. We will
> concentrate
> > on the "% of equity available, as it is most relevant for system
> development
> > and comparison.
> >
> > c. Trade Price -- the v7 tester allowed you to select the price
> field
> > (O,H,L,C) for entry and exit and the delay for each. In other
> words you
> > could have different delays for entry and exit.
> > The v8 tester does not allow different delays for entry and exit.
> It uses
> > the same delay for all entries and exits. The v8 tester also allows
> > different price fields for the long and short entries and exits,
> which the
> > v7 did not. But there is more you should know.... read on.
> >
> > The biggest difference in v8 is the addition of a button
> for "Realistic
> > Market Prices". It is important to understand what happens when
> you check
> > this button.
> >
> > To explain, let's assume you have "Position Limit" =1, "% of equity
> > available"=100 and the "Realistic Market Prices" button checked.
> >
> > When you get an enter Long signal, the program calculates the
> number of
> > shares that can be purchased using the open price on the signal bar
> (day0).
> > It tries to purchase that number of shares at the open price on the
> next bar
> > (day1). However, if the open price on the next bar (day1) is
> higher than
> > the price from the signal bar (day0), the tester does not place the
> order
> > because you do not have enough money. Now let's say you did not
> have enough
> > money. If your system is gives one day signals like you would get
> from a
> > cross(x,y) signal, then this trade will never be entered.
> >
> > If your system stays on a buy like you might have with a latch or
> with a
> > c>mov(c,20,e) structure, then the tester will still be on a buy
> signal on
> > day1. It will then recalculate the number of shares that can be
> purchase at
> > the open on day 1 and attempt to buy them on day2. If it can't buy
> because
> > the price is up at the open on day2 then it will not enter. The
> process
> > will continue until either the price drops at the open and you get
> filled or
> > the buy signal goes away. Thus it is again possible that you never
> enter on
> > this trade signal. The other possibility is that you will not get
> filled at
> > 100% of your equity.
> >
> > Now for a moment assume that you had your "Position Limit" set at
> 10, and
> > your buy signal remains in effect. The tester will continue each
> bar to try
> > to
> > fill your orders as above, and will add positions until either your
> buy
> > signal goes away, or your equity is used up, or your positions are
> equal to
> > 10. So if you have a buy signal that lasts for 20 bars, you may get
> 10
> > positions filled where the last one may be filled on the last bar
> before
> > the buy signal goes away.
> >
> > Note: the reverse happens on enter short signals.
> >
> > When the system gets an exit signal (either long or short) it exits
> on the
> > open of the next bar if you have "Realistic Market Prices"
> checked. Note
> > that there does not seem to be any issues on the exits. Since it
> is an
> > exit, it just exits regardless of the prices.
> >
> > If your system is an always in the market system, and you flip from
> a long
> > to a short, it will exit longs on the next bar, and attempt to sell
> short
> > the number of shares it calculates based upon the open price of the
> signal
> > day. If the price on day1 is higher, it does not enter the short
> sale.
> >
> > So, it is key to realize that if you use "Realistic Market Prices",
> > Your tests are not at all realistic because you may not get filled
> on the
> > day of the signal, or you may get filled at all, or you may get
> filled
> > multiple times. Implications are:
> > 1. Can't compare systems on same security.
> > 2. buy sell arrows represent entry/exit dates and not signal dates.
> > 3. This problem applies to all securities including mutual funds.
> > 4. Very difficult to develop systems and see valid signals.
> > 5. Can't compare performance on one security to another with same or
> > different systems
> >
> > Next let's review how the "Trade Price" from the "Trade Execution"
> screen
> > works if you do not have the "Realistic Prices" option set.
> >
> > The system calculates the number of shares to buy based on the
> price field
> > you choose and then tries to buy them on the price field you choose.
> >
> > If you have "Buy Price" = Open and Delay =1, the system calculates
> the
> > number of shares to buy based upon the Open price on the signal bar
> and
> > attempts to buy at the open of the next bar. If you have "Buy
> Price" =
> > Close and Delay = 1 the system calculates the number of shares
> based upon
> > the close of the signal bar and attempts to buy on the close of
> the next
> > bar. Same for Sell Price, Sell Short Price, etc. So if all you
> do is
> > uncheck the "Realistic Market Prices" button and use "Buy on open
> with
> > delay=1", nothing really changes.
> >
> > This is not logical at all! If you decided to buy on the open
> tomorrow
> > based on a signal today, only an idiot would calculate the number
> of shares
> > to buy based on today's open! There has been a whole day's worth
> of price
> > action transpired already and the open from yesterday
> > is no longer relevant. You would logically use today's close
> to "Estimate"
> > the number of shares that you "Might purchase" the next day.
> >
> > Because of all this it is difficult to develop and compare systems
> using the
> > v8 tester that produce valid results.
> >
> > Now, it gets even worse. All this tells me the optimization
> results are not
> > valid in v8. Since
> > all this buy, can't buy because price is up, etc etc stuff occurs,
> I assume
> > it occurs on each optimization run. So your optimization results
> are not
> > based on reality. In some trades, you don't get filled because of
> this. In
> > others you get filled late. A $.01 increase on the open would
> potentially
> > cause you not to be filled. So it makes the optimization results
> > inconsistent and unreliable. When you optimize you want to find
> the best
> > signals that optimize your performance. The potential non or late
> fills
> > because of a small price change on a given parameter run totally
> distort
> > this.
> >
> > So what can you do to get around this to develop and compare
> systems,
> > compare multiple systems, etc. ?
> >
> > Note that the above issues do not seem to effect you if you use
> a "Points
> > Only" test. In this mode apparently the v8 works just like the v7
> tester.
> > So you can get valid results that way.
> >
> > But if you wish to see an equity curve, or get a return in % then
> you have
> > the above problems.
> >
> > There are two ways to get around this and get valid results and
> signals.
> >
> > First, don't use "Realistic Market Prices".
> > Make sure your "Position Limit" is set to 1.
> > Set Equity Default size to 100%.
> > Under Margin Requirements to
> > Long Initial = 100%
> > Long Maintenance = 0%
> > Short Initial = 200% (amazingly, 200% means no margin, but
> that is
> > another story)
> > Short Maintenance = 101% (this means essentially no
> maintenance. for
> > some dumb reason they won't let you put in 100%, so you will never
> be able
> > to trade your full equity on a short trade, only 99%.)
> >
> > Now you have two options.
> > 1. Set all your enter and exit positions to "Close" with a Delay =0.
> > This will enter and exit all of your equity (except for the stupid
> 1% limit
> > on short sales) at the close. All your signals will show up on the
> charts
> > correctly, your % gains will be correct and it will produce results
> very
> > close to the v7 tester. Of course the drawback is that in the real
> world,
> > you probably won't enter and exit on the same bar as the signal.
> However, if
> > you are mainly looking to compare systems or see signal dates,
> then this
> > works.
> >
> > 2. Set all of your enter and exit conditions to use format of
> > Ref(enter,-1).
> > Use "Price Field = Open with Delay =0". (or Close with delay of 0,
> or
> > whatever).
> > What happens is that the condition that is true on day0 gets
> evaluated as
> > true on day1 (the day after the signal and the right day to
> enter). The
> > number of shares are calculated and purchased at the same price.
> The arrows
> > show up on the chart on the entry/exit day.
> >
> > This is as realistic as the v7 tester and actually pretty good.
> > The downside is you have to rewrite all your systems to use the Ref
> ()
> > function in all the enter and exit conditions. Lots of work and
> pain.
> >
> > Using either of these methods should make the system tests valid,
> match 7.2
> > results, and make the optimization results valid.
> >
> > What should be done is that Equis changes the tester. Here is what
> I have
> > recommended they do:
> >
> > If the "Realistic Market Prices" button is checked the system
> should adjust
> > the number of shares to buy everything allowed by the equity in the
> account
> > at the open the next day.
> >
> > When the user selected options for Buy,Sell, etc are checked, the
> system
> > should adjust the number of shares to buy the total $ allowed by
> the equity
> > at the price field selected, on the bar selected by the delay field.
> >
> > Of course the other option is to just put it back to the v7 method.
> >
> > One other caveat, I have done any testing attempting to use the
> limit
> > orders so I have no idea what happens when you try to do that.
> >
> >
> >
> > ----- Original Message -----
> > From: "equisdvc" <david_cardner@xxxx>
> > To: <equismetastock@xxxxxxxxxxxxxxx>
> > Sent: Sunday, November 23, 2003 7:53 PM
> > Subject: [EquisMetaStock Group] Bugs in v8.01
> >
> >
> > > I have been working with the Equis Technical Support personnel
> since
> > > I purchased v7.0 in June of 2000. The support has been
> excellent, so
> > > when v8.0 was released, I bought it. I had developed my own
> trading
> > > algorithm and was particularly interested in being able to
> calibrate
> > > it against a portfolio of equities with the System Tester.
> However,
> > > I found the number of bugs in v8.0 was overwhelming and I had to
> > > revert back to using the v7.22 System Tester. My personal
> observation
> > > was that version 8.0 was not even qualified for alpha testing at
> the
> > > time of its release.
> > >
> > > I was delighted when v8.01 was released six months later. I
> thought
> > > the bugs had been corrected. I was wrong. I discovered that
> v8.01,
> > > although a big improvement, would barely qualify as a beta test
> > > version. I was both upset and embarrassed because I had talked
> two
> > > friends into buying v8.01.
> > >
> > > With both v7.22 and v8.01 on the same computer it is easy to
> compare
> > > System Tester results. The results between the two versions
> running
> > > the identical algorithm are drastically different, regardless of
> > > whether they are being run in normal or points-only mode. This
> occurs
> > > when v8.01 has been carefully configured to reproduce v7.22
> results.
> > > Bottom line is that v7.22 results make sense, v8.01 results do
> not.
> > > Technical support has not been able to explain the differences,
> nor
> > > provide a workable solution. I still do not use v8.01. I use only
> > > v7.22 because it is the only version that I trust.
> > >
> > > What I find unusual is that I can't find any other reference on
> the
> > > internet to the mismatch between v7.22 and v8.01 System Tester
> > > results. In my 43 years of using computers for problem solving,
> and
> > > having written, debugged, and documented roughly 100,000 lines of
> > > code, one of the primary requirements when modifying programs is
> that
> > > they must be able to correctly reproduce previously validated
> > > results. This test was obviously not applied to v8.0 or v8.01.
> Yet
> > > it is the only way to validate the changes made to the program.
> > >
> > > Is there anyone out there that encountered this situation and
> > > understands its significance?
> > >
> > > David
> > >
> > >
> > >
> > >
> > > To unsubscribe from this group, send an email to:
> > > equismetastock-unsubscribe@xxxxxxxxxxxxxxx
> > >
> > >
> > >
> > > Your use of Yahoo! Groups is subject to
> http://docs.yahoo.com/info/terms/
> > >
> > >
>
>
>
>
> To unsubscribe from this group, send an email to:
> equismetastock-unsubscribe@xxxxxxxxxxxxxxx
>
>
>
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
>
>
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Buy Ink Cartridges or Refill Kits for your HP, Epson, Canon or Lexmark
Printer at MyInks.com. Free s/h on orders $50 or more to the US & Canada.
http://www.c1tracking.com/l.asp?cid=5511
http://us.click.yahoo.com/mOAaAA/3exGAA/qnsNAA/BefplB/TM
---------------------------------------------------------------------~->
To unsubscribe from this group, send an email to:
equismetastock-unsubscribe@xxxxxxxxxxxxxxx
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
|