Re: Bob Fulks or anyone

[Bob Fulks <bfulks@xxxxxxxxxxxx>]

At 3:07 PM -0800 2/6/02, Ray Gurke wrote:

>Would also appreciate hearing more on what folks are doing in regard to
>bad-tick filters, as a wild tick could really throw your calcs/signals
>haywire...

I have never found a good general purpose solution to the bad-tick
problem. Ideally, they should be filtered out by the Global Server in
TS2000i but they aren't.

If you are looking at the charts before taking a trade, to see it
they look "normal", then you can be more tolerant. But if you are
executing trades automatically, you obviously have to be much more
careful.

If your system depends upon only the Close of a bar, bad ticks are
not usually a big problem since there is a small probability that the
bad tick will be at the Close of the bar. But if you use the High or
Low of a bar, these will always be affected so you will need a filter.

You can usually re-write your system to work on higher resolution
bars and do the filtering there. For example, if your system works on
30 minute bars, you can re-write it to work on 5 minute bars and
still execute the same trades, either by building "synthetic
30-minute bars" or by using every 6th bar. It is tricky but usually
be done. Then, you can look at the 5-minute bars and filter out bad
values before passing the bars on to the 30 minute system.

You can even build the system to use 1-tick or N-tick bars to get
even higher resolution bars but this will require a lot of computing
power on high-volume symbols such as ES or QQQ. It does work well on
SPX and NDX where you get four ticks per minute.

As for the actual filtering, the ideal filter would probably filter
out ticks that are greater than three or four standard deviations
away from an average. This requires complex calculations so you can
usually get by with using a multiple of the mean average deviation.
You also have to account for gaps between days on an intraday chart.

To actually account for sudden price changes, you could keep a count
of the number of consecutive rejected bars so that if that number
exceeds some number of bars, then you reset things. The following
code fragment will give you the basic idea:


if Date <> Date[1] or                    { on a new day or       }
   bCount > bCntLim then begin           { bad count > limit     }
      Avg = High;	                 { initialize average    }
   end;

Dev = AbsValue(High - Avg);              { calculate deviation   }

if Dev < Mult * MDev then begin          { if not a bad tick     }
   bCount = 0;                           { reset bad tick counter}
   xHigh = High;                         { use value             }
   Avg = Avg + factor1 * (xHigh - Avg);  { update average        }
   MDev = MDev + factor2 * (Dev - MDev); { update mean deviation }
end else bCount = bCount + 1;            { inc bad tick counter  }


Hope this is useful.

Bob Fulks