|
PureBytes Links
Trading Reference Links
|
-------- Original Message --------
Subject: Fw: Ts2ki GS files and Diagnostics (10K GIF)
Resent-Date: Thu, 12 Mar 2009 13:37:21 -0700
Resent-From: omega-list@xxxxxxxxxx
Date: Thu, 12 Mar 2009 13:37:18 -0700
From: RL Scott <rlscott1@xxxxxxxxxxxxxx>
Reply-To: RL Scott <bob.scott.56@xxxxxxxxx>
To: Omega-List <omega-list@xxxxxxxxxx>
I have just witnessed a sight that I never thought I would see and that was
a Ts2ki file trying to exceed the 4 GB limit. The file did not stop
updating, it just disappeared and if there was no backup all 4 GB of that
data would be lost. The way Ts2ki works is that whenever a new symbol is
added the program assigns it to a particular GS data file and all data for
that symbol must be stored in that specific file and can't be spread across
files. If you have a lot of tick data you may be approaching a similar
situation with your system. I have just written a freeware program which
when the GS is shut down will determine how much available space is in the
file (see attached image) and if the available space is getting low the
report shows where the most data is stored. For instance, on my very small
program development GS data even though the "Available-Space" is large, it
is apparent that Citigroup (C) tick data is by far the largest storage
consumer in GS File 00000010.DAT and if I needed to reduce data size then
reducing the number of days of "C" tick history might be a good choice.
When GS data is deleted, the size of the GS file does not get smaller but
that deleted data space is available for other data storage. The only way
to reduce the size of GS files to the minimum is to use a third party
program such as the Ts2ki Unfrag utility. In the case of the subject file,
only 3 KB was available free space so even the Ts2ki Unfrag utility would
not have helped here - the file just had too much data.
What should have been done was to delete data until there was sufficient
available space. The problem - which symbols are stored in which files -
and until I wrote the Ts2ki Diagnostics program that information was not
available and finding the proper symbol for data deletion was similar to a
"crap shoot". Once sufficient space was available in the file (I like to
see at least 800 MB) then the Ts2ki Unfrag Utility could have been used to
defragment the file from the almost 440,000:1 Fragmentation Factor to 1:1.
I have provided the Ts2ki Diagnostics utility free of charge on my website
at http://www.bobscottsystems.com in the Downloads section.
I would advise anyone with significant amounts of GS data to use this
utility.
Best regards,
~Bob
|