|
PureBytes Links
Trading Reference Links
|
The issue concerning the first bad tick and its effects on the
intraday data has generated some regrettably uninformed yet
patronizing statements on this list. In the absence of any
information on the subject from Omega, let me offer some of
my observations that hopefully some members may find helpful.
The principal cause of the editing difficulties, associated with the
initial tick, is the fact that TS does not save all incoming ticks at the
original value. The value with which TS saves and eventually displays
a bad tick depends on two factors: (1) How much off the ticks is,
and (2) What the tick's position is relative to the first tick in the
trading session.
Most bad ticks are the "what you see is what you get" kind of ticks.
They are usually off by a small margin -- say, a point or two. Let's
call them "the small margin ticks." TS saves them at their face value.
Therefore, it does not make any difference where they are relative
to the initial tick. They can be easily corrected with no effect on
other ticks received during the session.
Then there is another category of bad ticks -- the ones that are off
by a big margin. For some eerie reason, TS stores each tick that
is off by more than about 33 points -- 330 points for high priced
symbols -- at a value that is different from the original value. (I think
the threshold is closer to 32.817 -- or 328.17, as the case may be
-- but I do not have time to figure that out. Those who are curious
about the algorithm used by Omega and its rationale should
question Omega techsupport.)
Now, about the location of a tick.
As I said, small margin ticks are no problem. The same goes for
big margin ticks if they are NOT in the first position in ANY group
of 60 ticks, starting from the beginning of the session. Ticks like
these are saved with different values but they do NOT affect other
ticks collected in the session. Therefore, they also can be easily
corrected or deleted.
The big margin ticks, heading the blocks of 60, are treated
differently. TS saves them at the face value even though they
may be off by hundreds of points. However, TS will alter
the value of ALL 59 remaining ticks in its group, while the
diferentials between individual ticks are maintained. Ticks in
other groups remain unchanged.
Here are some examples (you can check them in the Tick Editor):
IBM, Jan 9, 98
....................................................Saved by TS as
Tk.No...............1..............2..............1............2
OrigValue....104.312....104.312
Change to.....................71.312....................136.848
..................................137.312......................71.776
....................71.312.....................71.312.......38.776*
..................137.312...................137.312.....169.848*
DSP8H Jan 9, 98 (Note that for SP8H the initial tick is the
first tick recorded after midnight, if Globex hours are enabled)
....................................................Saved by TS as
Tk.No...............1..............2.............1.............2
OrigValue.....960.50... ..960.00
Change to....................630.50....................1285.86
.................................1290.50......................635.14
...................630.50.....................630.50.......304.64*
.................1290.50..................1290.50......1615.36*
(* - when the first tick is changed, so are the ticks No. 2-59.)
Note that when the first tick is off by more than 33 points on the
downside, TS usually computes the value of the next tick is by
deducting approx. 33 points (or 330) from the value of the bad
tick. If it is on the upside, the 33 points are added. If a bad tick
hits any position between No.2 and No. 59, the process is
reversed.
Thas was a simplified demonstration of the way TS behaves in a
static environment, when one looks at the data already collected
and stored on the hard disk.
However, under dynamic conditions -- during market hours,
when live ticks keep coming in -- things are somewhat different:
If a big margin tick hits the first position in any group of 60,
ALL ticks that follow -- not just the next 59 ticks -- will be changed.
The probability of this happening is low but it does happen. We
saw it a few months ago with some indexes such as INDU, OEX
and SPX when the problem appeared in the afternoon trading.
The big margin ticks received before the beginning of trading
cause the greatest damage because they alter the complete file.
What makes this a serious problem is the fact that such a tick
does not have to be the first tick of the trading session. It can be
ANY big margin tick received by TS between midnight and 9:30.
The probability of getting a piece of garbage from a datafeed --
especially, if there is a reception problem -- during the 8.5 hour
period is significantly greater than getting a single large-margin
bad tick right at the start of trading. If a bad tick comes in before
the opening, TS will treat it as the initial tick, while converting all
subsequent ticks, including the first tick of the trading session,
to "bad data."
I have received such bad first or out-of-session ticks many
times from the BMI feed. For instance, over the past 30 days,
the following symbols in my portfolio -- not counting SPX on
Jan 5 -- had an entire day of data altered by the initial tick
(the bad data are present in Omega's refresh files):
Date......Symbl...Tk.No...Time......Value... Correct Price Level
Dec 11.....CI........1.......09:46....128.125...........160
...........................2.......09:48....100.464
Dec 31....BNI.......1.......01:15........0.001............92
...........................2.......03:23........0.000
...........................3.......09:37......27.464
..............DAL......1....... 02:36...... -0.084..........115
...........................2........09:30.....-15.197
Jan 2.......K.........1........08:57........0.677............50
...........................2........09:31....-16.161.
Jan 5.......CI........1........09:43......17.250...........170
...........................2........09:44......41.303
.............INTC......1........09:28........0.040............73
...........................2........09:30.......7.526
..............WY.......1.......09:32.....-375353.552......50
...........................2.......09:32.....-375341.458
This was also the case with INTC on Jan 5. As Rod Grisham
wrote to me, his computer received the tick of $0.04 at 9:28am.
As a result, the next tick, which was the first tick of trading, was
sent by BMI, apparently, at the correct price of 75.062 but it
was saved by TS at 7.526. All ticks that followed throughout the
session were similarly converted to the price levels set by the
bad tick. The resulting file of 18,840 ticks contained 280 ticks
-- each of them 59 ticks apart. Once one of these 280 ticks
was corrected, TS restored the correct value of the remaining
ticks in the respective tick's group. That such regularity can not
be a consequence of "a series of datastream errors" -- as Mr.
Brickley has been suggesting -- should be obvious to anyone
with scientific training.
The recent ITNC incident demonstrated that Omega techsupport
is not equipped to handle the damage set off by a single first or
out-of-session bad tick. With no reasonable way of restoring
corrupted data, Omega users should be concerned about the
possibility that even a small problem with a datafeed could affect
many symbols for several days and, in effect, render their
intraday data useless.
Under the circumstances, it would be plain silly not to demand
from Omega a simple utility that would only repair what Omega's
software has previously mangled. If Omega's managers were
smart, they would be grateful for the prodding received from
Rodney Grisham and me to make TS a functional product. The
same goes for some members of this list.
IU
|