Re: [amibroker] Re: Time stamping an array

I am again and again amazed at how some of the common problems we struggle with for many hours (months, years) without success, can be solved, in what looks like a relatively easy way, by an expert programmer (like you). I have noticed that lately you are publishing more coding solutions and this is very much appreciated. Your programming solutions are extremely helpful, they may be simple to you but I bet that 90% of AB users would be unable to write the code.

Hello,

1 & 2. It is possible to create new one-line file (consisting of one new bar) and import it every 5 second. To import

use OLE automation Import() method.

In the future I will however provide a faster "internal" method of padding data so you will get equally spaced chart regardless

if there are data within given period or not.

Best regards,

Tomasz Janeczko

amibroker.com

----- Original Message -----

From: Herman

To: Tomasz Janeczko

Cc: amibroker@xxxxxxxxxxxxxxx

Sent: Saturday, December 29, 2007 10:17 PM

Subject: Re: [amibroker] Re: Time stamping an array

thanks Tomasz,

1) Is it possible to create and import this file automatically at 5 seconds intervals?

2) Instead of creating the entire file each time would it be possible to just add a new bar to it at 5 sec interval?

...would this make a nice post for your KB?

seasons greetings,

herman

For tips on developing Real-Time Auto-Trading systems visit:

http://www.amibroker.org/userkb/

Saturday, December 29, 2007, 3:20:46 PM, you wrote:

> Hello,

> You can create perfect reference ticker easily using ASCII import.

> Just create the text file using format:

> DATE,TIME,O,H,L,C,V

> and import it

> Such text file can be created programmatically very easily

> Simple example (for simplicity it creates 24 hour (non-stop) 5-second data bars

> for entire January 2007.

> fh = fopen("refticker.txt", "w");

> mo = 1; yr = 2007;

> for( d = 1; d <= 31; d++ )

> for( hr = 0; hr < 24; hr++ )

> for( m = 0; m < 60; m++ )

> for( s = 0; s < 60; s += 5 )

> {

> line = StrFormat("%04.0f-%02.0f-%02.0f,%02.0f:%02.0f:%02.0f,1,1,1,1,0\n", yr, mo, d, hr, m, s );

> fputs( line, fh );

> }

> fclose( fh )

> Generated file has 16 Megabytes and it takes about 3 seconds to run the above formula.

> Best regards,

> Tomasz Janeczko

> amibroker.com

> ----- Original Message -----

> From: "Dennis Brown" <see3d@xxxxxxxxxxx>

> To: <amibroker@xxxxxxxxxxxxxxx>

> Sent: Saturday, December 29, 2007 7:01 PM

> Subject: Re: [amibroker] Re: Time stamping an array

>> Eric,

>> I just need one "Perfect" reference ticker that covers my whole time

>> span. I am not the only one who needs this.

>> I am working on day trading futures like ES with indicators of my own

>> design. I use a 5 second database so that I can get range bar and

>> volume bar charts in the 2-4 minute average bar time. A tick

>> database would take too much storage and time to process. I trade

>> from the resultant charts. In order to optimize my parameters, I

>> need 30 days of data. However, the contracts expire every 3 months

>> and must be rolled. Data suppliers usually provide a continuous

>> contract ticker that switches to the next months contract and

>> prepends adjusted prices of the older contract to give a smooth

>> transition to the rollover for indicators.

>> However, THIS DATA IS FLAWED and my charts are ruined for 5 weeks

>> each quarter.

>> First, most suppliers only provide 8-10 calendar days of backfill

>> data for 5 sec data (which is actually aggregated tick data at the

>> local level). Since back prices are adjusted in the continuous

>> contract, you can not simply collect 200K bars with AB and have a

>> complete seamless 30 day database. When the contract rolls, the old

>> prices in the database will not be adjusted and create a seam. A

>> forced backfill to fix this will just move the bad seam 8-10 days back.

>> Second, for about a week or two, the volume gets split between the

>> old contract and the new contract as traders use different criteria

>> to roll to the new contract. The supplied continuous contracts

>> usually roll when the volume of the new contract is higher than the

>> old contract. They simply switch contracts in the quotes. This has

>> the effect of having the volume of the continuous contract look like

>> it drops over a week to half the usual volume, then rises up again

>> over the next week. However, if you look at the sum of the volumes

>> for the two contracts, you see that it remains about the same as

>> usual. This ruins constant volume charts and other volume based

>> indicators.

>> To solve both of these problems, I am making my own 30 day continuous

>> contracts in AB. I collect the data from the individual contracts

>> with no adjustments. Then I fill an OHLC array with one contract or

>> the other depending on when the rollover criteria is met and adjust

>> the old data to get rid of the price seam (like adjusting for

>> dividends or splits). Finally I add the volumes of both contracts

>> together for the final 5 second ticker and write it out with ATC.

>> This creates a nice ATC ticker to use in another chart.

>> I then have another chart (my actual main trading chart) that reads

>> the ATC ticker and works in the timeframe desired for running

>> indicators and trading.

>> I should point out that writing out an ATC ticker every 5 seconds is

>> slow. I actually only write it out when the main chart timeframe

>> calls for a new bar which is once every 2-4 minutes. In the mean

>> time, I just send the OHLCV of the current bar to the main chart to

>> keep the realtime signals going.

>> Right now, I have to use a supplied continuous contract as a

>> template. However, because it is not seeing all the bars from both

>> contracts, some of the volume has no bar to be registered in. The

>> after hours loses a lot of bars. It also make switching to different

>> futures a pain, because I have to switch the ATC ticker of the main

>> chart, then, select the continuous contract chart and select the

>> corresponding ticker there, then back to the main chart. Not the

>> easy "just click on the ticker" to get the next chart.

>> I just need one "Perfect" reference ticker that covers my whole time

>> span. Then the continuous contract generating chart can stay out of

>> sight and out of mind.

>> Best regards,

>> Dennis

>> On Dec 29, 2007, at 10:40 AM, eric tao wrote:

>>> Could you give a sample of what you were trying to do?

>>> BR

>>> --- In amibroker@xxxxxxxxxxxxxxx, Dennis Brown <see3d@xxx> wrote:

>>>> Herman,

>>>> Thank you again for you reply. If what you say is true, then we need

>>>> Tomasz to provide a solution to this problem. It is such a

>>>> fundamental need for some of us, and workarounds do not seem

>>>> adequate.

>>>> Best regards,

>>>> Dennis

>>>> On Dec 29, 2007, at 4:55 AM, Herman wrote:

>>>>> If you find one, please post your solution. Right now the only

>>>>> solution is to create an array in Excel and import it each 5

>>>>> sec this is a rather daunting task in real-time. Tomasz

>>>>> acknowledged the key problem in the AB Help for the foreign()

>>>>> function:

>>>>> "Please note that if you have data holes in currently selected

>>>>> symbol then in order to synchronize bars Foreign function will

>>>>> remove bars that exist in Foreign symbol but do not exist in

>>>>> currently selected symbol."

>>>>> This unscientific approach makes the accuracy of all TA analysis

>>>>> depend on the underlying ticker. It assumes a perfect database

>>>>> which, in EOD is rare, and in real time simply does not exist.

>>>>> What is needed is a real-time reference array that can be made

>>>>> current and contains bars for all time periods, i.e. creates empty

>>>>> bars in real-time when data doesn't exist.

>>>>> best regards,

>>>>> herman

>>>>> For tips on developing Real-Time Auto-Trading systems visit:

>>>>> http://www.amibroker.org/userkb/

>>>>> Friday, December 28, 2007, 11:59:48 PM, you wrote:

>>>>>> Hello,

>>>>>> Forgive me if it seems like I am restarting a thread out of my

>>>>>> original, but the previous thread got badly broken when Yahoo

>>>>> dropped

>>>>>> a number of replies, so I thought to try again fresh.

>>>>>> I really need a reference ticker of 200k 5 second 24 x 7 bars. I

>>>>>> need it as a template to make continuous contract futures data that

>>>>>> RT charts can use.

>>>>>> For a reference ticker, all the data fields OHLCV OI can be set to

>>>>>> one (1). The only data that I need to be able to change is the

>>>>>> time

>>>>>> stamp. Once made, every 5 seconds, the reference ticker would be

>>>>>> read, all the bars shifted one bar, and the last bar time stamp

>>>>>> increased by 5 seconds, then written back out (ATC). This is a bit

>>>>>> of an over simplification, but you get the idea.

>>>>>> The only thing I don't know how to do is rewrite the time stamp

>>>>>> on a

>>>>>> bar.

>>>>>> I would appreciate any specific ideas as to how I could do this

>>>>>> --or

>>>>>> another way to get my continuous 5 second RT perfect reference

>>>>>> ticker. Perhaps there is a dll way? Speed of maintaining the RT

>>>>>> ticker is important.

>>>>>> I have tried many different approaches for a couple of weeks and

>>>>>> end

>>>>>> up with a roadblock with each way so far. Herman has discussed the

>>>>>> problems in the UKB, but none of the solutions so far will work

>>>>> for me.

>>>>>> Best regards,

>>>>>> Dennis

>>>>>> Please note that this group is for discussion between users only.

>>>>>> To get support from AmiBroker please send an e-mail directly to

>>>>>> SUPPORT {at} amibroker.com

>>>>>> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:

>>>>>> http://www.amibroker.com/devlog/

>>>>>> For other support material please check also:

>>>>>> http://www.amibroker.com/support.html

>>>>>> Yahoo! Groups Links

>>> Please note that this group is for discussion between users only.

>>> To get support from AmiBroker please send an e-mail directly to

>>> SUPPORT {at} amibroker.com

>>> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:

>>> http://www.amibroker.com/devlog/

>>> For other support material please check also:

>>> http://www.amibroker.com/support.html

>>> Yahoo! Groups Links

>> Please note that this group is for discussion between users only.

>> To get support from AmiBroker please send an e-mail directly to

>> SUPPORT {at} amibroker.com

>> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:

>> http://www.amibroker.com/devlog/

>> For other support material please check also:

>> http://www.amibroker.com/support.html

>> Yahoo! Groups Links

> Please note that this group is for discussion between users only.

> To get support from AmiBroker please send an e-mail directly to

> SUPPORT {at} amibroker.com

> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:

> http://www.amibroker.com/devlog/

> For other support material please check also:

> http://www.amibroker.com/support.html

> Yahoo! Groups Links

> <*> To visit your group on the web, go to:

> http://groups.yahoo.com/group/amibroker/

> <*> Your email settings:

> Individual Email | Traditional

> <*> To change settings online go to:

> http://groups.yahoo.com/group/amibroker/join

> (Yahoo! ID required)

> <*> To change settings via email:

> mailto:amibroker-digest@xxxxxxxxxxxxxxx

> mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx

> <*> To unsubscribe from this group, send an email to:

> amibroker-unsubscribe@xxxxxxxxxxxxxxx

> <*> Your use of Yahoo! Groups is subject to:

> http://docs.yahoo.com/info/terms/