Re: [amibroker] Re: System Performance Indicators [was: Can someone fix this OLE code?], AmiBroker Email List Archive

I only trade one type of future contract at a time, so it is "easy" to write any performance metric I want, including an array of a normally single value type metric over time. I am a simple person, so I like to keep things simple so I can understand them.

BR,

Dennis

On May 18, 2008, at 3:51 PM, Fred Tonetti wrote:

… and you get performance metrics from an indicator how ? …

From: amibroker@xxxxxxxxxxxxxxx [mailto:amibroker@xxxxxxxxxxxxxxx] On Behalf Of Dennis Brown
Sent: Sunday, May 18, 2008 3:45 PM
To: amibroker@xxxxxxxxxxxxxxx
Subject: Re: [amibroker] Re: System Performance Indicators [was: Can someone fix this OLE code?]

Please pardon my jumping in here with a comment. I don't really belong in this thread, because I wrote everything I need for backtesting my systems in AFL only (yes, in indicator mode), so I can of course write whatever I need in AFL.

However, it seems to me that there are just two basic philosophical concepts that need to be clarified.

1. Is it reasonably possible to do what you want with the tools provided?

2. Is it easy to do what you want with the tools provided?

I contend that if it is possible, then it can be made easy without changing the tools. The missing ingredient is simply someone who understands how to do it, leading the way by creating the example for others to follow.

I may not understand if something is possible, hard, or easy when I look at what I want to accomplish. After looking around for examples, I may conclude that it is one of those three cases. If it is easy, it is because I found an example that is close to what I want. If it is hard, it is because I found an example that is far from what I want, but gives me some idea about out to approach the problem. If I can find no examples at all, I am left with a question if it is even possible and possibly a waste of my time trying to do it. That is when I ask for advice here about the possibility, hoping for the unfortunately rare definitive answer one way or the other.

If something that is not currently possible is needed to make money with the system (and yes, sometimes UI things that might be considered just aesthetics to an EOD person do fall into that category for day traders who have only one second to take an action or lose money), It should be addressed by TJ in AB with some priority.

If it something that is possible, but not easy, then it needs to be addressed by creating examples of how to do it, or even creating #include samples that make it as easy to deal with as if it were built-in. This can be done by TJ, or by other advanced users and made available in some convenient place like the UKB.

It is important to understand which category a problem falls into so that the appropriate remedy can be pursued. We will get the most out of AB by being clear about this and making suggestions for additions to AB/AFL where no good alternative exists, and asking for good, well explained examples where the solution exists, but is not obvious.

It seems that this thread started with a lack of clarity of what exactly is needed first, and how to get the desired result from AB second. The result is several frustrated people not connecting with a common understanding of the problem much less a solution.

(Yes, I get frustrated also, when I don't seem to be able to communicate my needs in a way that results in good solutions.)

If you are still reading, thanks for listening to my rant.

Best regards,
Dennis

On May 18, 2008, at 1:55 PM, Fred Tonetti wrote:

PS …

… and you REALLY don’t want to try this in AFL …

AB = CreateObject("Broker.Application");
AA = AB.Analysis;
AA.Backtest(0);
AA.Export("Trades.CSV");

… The reason of course is that whether you try and run a backtest with this AFL or just click the check mark to verify the AFL you’ll cause AB / AA to get into an infinitely running backtest i.e. a backtest which runs a backtest which runs a backtest ad infinitum …

However, if instead you create a .VBS file with this in it …

Set oAB = CreateObject("Broker.Application")
Set oAA = oAB.Analysis
oAA.Backtest(0)
oAA.Export("Trades.CSV")

Then the resulting Trades.CSV file will have what is expected in it.

I guess I have for a fair amount of time assumed that if what you need is directly available in AB, like ~~~Equity, then you can manipulate it directly in AFL … If what you need to get at is not directly accessible data, like a Trade List … Then you need a script to do this and by definition that script needs to be external to AB.

While I suppose it is conceivable to have an AFL that runs an EXTERNAL script which in turn runs a backtest and makes the results of what it does available to the calling AFL not only is that at least at first glance overkill and overly complicated it probably also fails for one reason or another …


From: amibroker@xxxxxxxxxps.com [mailto:amibroker@xxxxxxxxxps.com] On Behalf Of Fred Tonetti
Sent: Sunday, May 18, 2008 1:34 PM
To: amibroker@xxxxxxxxxps.com
Subject: RE: [amibroker] Re: System Performance Indicators [was: Can someone fix this OLE code?]

As a followup …

If you for example look at the AFL below, what it should produce and what it actually does produce you’ll see that:

-          The equity curve that results from running a backtest gets exported as one would think in should.
-          However, the output Trade List which is also a byproduct of running a backtest only exports the column headings when one attempts to do it via what would be OLE/Automation commands that more appropriately belong in something external to AB.

This is not a complaint per se, it’s simply what is …

Len1 = Optimize("Len1", 13, 1, 100, 1);
Len2 = Optimize("Len2", 23, 1, 100, 1);

EMA1 = EMA(C, Len1);
EMA2 = EMA(C, Len2);

Buy = Cross(EMA1, EMA2);
Sell = Cross(EMA2, EMA1);

OPEquity = Foreign("~~~Equity", "C");

Beg = LastValue(ValueWhen(Status("FirstBarInRange"), BarIndex()));
End = LastValue(ValueWhen(Status("LastBarInRange"), BarIndex()));

yyyy = Year();
mm   = Month();
dd   = Day();

fh   = fopen("C:\\Program Files\\AmiBroker\\Equity.CSV", "w");

fputs("Date,Equity,\n", fh);

for ( i = Beg; i <= End; i++ )

{
    Line = "";
    Line = Line + StrFormat("%02.0f", mm[i]) + "/" + StrFormat("%02.0f", dd[i]) + "/" + StrFormat("%4.0f", yyyy[i]) + ",";
    Line = Line + StrFormat("%.5f", OPEquity[i]) + ",";
    Line = Line + "\n";
    fputs(Line, fh);
}

fclose(fh);

AB = CreateObject("Broker.Application");
AA = AB.Analysis;
AA.Export("Trades.CSV");

From: amibroker@xxxxxxxxxps.com [mailto:amibroker@xxxxxxxxxps.com] On Behalf Of Fred Tonetti
Sent: Sunday, May 18, 2008 12:23 PM
To: amibroker@xxxxxxxxxps.com
Subject: RE: [amibroker] Re: System Performance Indicators [was: Can someone fix this OLE code?]

I see both sides of the discussion so maybe I can add something to bridge the gap …

While I agree with TJ that, as presented in the backtest report or AA in an optimization output line, system performance metrics are scalars i.e. single numbers as of the end of some backtest or some single combination of parameter values from an optimization, one could via the symbol ~~~Equity look at these as an array of values that can be evaluated bar by bar …

For example Net Profit has a value all along the array represented by the ~~~Equity symbol.

If one also needs to be able to have available Trade based performance metrics then one would also have to have available to them the list of Trades from a backtest so that as of some particular bar one could calculate whatever performance metric(s) one was interested in …

There are undoubtedly many uses for this sort of information one of which I touched on briefly the other day i.e. When during the OOS period should we programmatically decide that our system needs to be reoptimized and as result WF our IS to that point and do so …

While I have done these sorts of things that are described above in IO and other places, I would add that I have not for the most part done them directly in AFL because as TJ states you may very well not get what you are looking for.

If this is the kind of thing you are after it is not that difficult to do …

From an external script it is relatively simple to:

-          Run a backtest … This yields the Trade List which can be exported from AA to a CSV file.
-          It also yields the equity curve in the symbol ~~~Equity which can be easily exported via a “manufactured” AFL i.e. have the script create the AFL, load it and run it.
-          This then provides in two CSV files everything that one needs to calculate any and all performance metrics one is interested in bar by bar.
-          The results of these calculations can be used to drive indicators in AB or as all or part of a decision making process about what actions to perform in some other type of AFL in AB.

From: amibroker@xxxxxxxxxps.com [mailto:amibroker@xxxxxxxxxps.com] On Behalf Of Tomasz Janeczko
Sent: Sunday, May 18, 2008 10:58 AM
To: amibroker@xxxxxxxxxps.com
Subject: Re: [amibroker] Re: System Performance Indicators [was: Can someone fix this OLE code?]

You guys don't get it at all.

Most metrics are SCALARS (single number) !
Your plots will look like this:

Plot( 0.8, "Sharpe Ratio", colorRed );

These metrics DO NOT CHANGE over time because they are all calculated ONCE when backtest is complete
and result is single number (scalar), *not* array.

Best regards,
Tomasz Janeczko
amibroker.com
----- Original Message -----
From: Phsst
To: amibroker@xxxxxxxxxps.com
Sent: Sunday, May 18, 2008 3:36 PM
Subject: [amibroker] Re: System Performance Indicators [was: Can someone fix this OLE code?]

Herman,

Your case for System Performance Indicators is a good one.

A further idea... there should be no need for User code to accomplish this. The plots should be a native feature of AB (just like the Equity plot).

Instead of plotting "flat lines" for single numbers, gfx Horizontal plots of each performance metric could be presented against a "good / bad scale" defined in the form of a Param value for each performance metric that the User could alter from a base set of default values. Of course the gradient colors of the plots would go from red (bad) to green (good).

For example, the default Param for Sharpe Ratio could be 1.00, and some Users might want to use .80 as a Param value where the gradient color starts changing to green.

My two cents worth.

I am using the free version of SPAMfighter for private users.
It has removed 455 spam emails to date.
Paying users do not have this message in their emails.
Try SPAMfighter for free now!

I am using the free version of SPAMfighter for private users.
It has removed 455 spam emails to date.
Paying users do not have this message in their emails.
Try SPAMfighter for free now!

I am using the free version of SPAMfighter for private users.
It has removed 455 spam emails to date.
Paying users do not have this message in their emails.
Try SPAMfighter for free now!

I am using the free version of SPAMfighter for private users.
It has removed 455 spam emails to date.
Paying users do not have this message in their emails.
Try SPAMfighter for free now!