Backtesting Actual Price Data

[ronald3rg]

Hello Everyone,

I was wondering if anyone was familiar with a way to backtest the actual market data as opposed to what TS2 shows on the chart. As you may already know the system is hiding price gaps so when you run a backtest and the strategy opens a position based on the last closed candle it may not actually be how it would play on out on the market.

This is more geared to the higher time frames specifically around the open of the market.


Also on the same note am trying to figure out how to make the strategy/strategies I currently use to open/start at the beginning of each new H8 or D1 candle based on the current market condition.

Currently I have to manually start and Pause the live candle setup of strategy which calls on this code.

Code: Select all


if OrderTime == "immidiately" then
        source = ExtSubscribe(1, nil, "t1", instance.parameters.Type == "Bid", "tick");
    else
        source = ExtSubscribe(1, nil, instance.parameters.TF, instance.parameters.Type == "Bid", "bar");
    end


After position is open I pause it so it does not jump back in if position is closed. Am testing various scenarios to see what works but wanted to see if anyone had any ideas. Backtesting these changes doesn't help because system always backtests and opens on new candle because of what I mentioned above.

However when put on live or demo account you get error message:: "Open order failedThe command is disabled."

ps: I do not have a FIFO account.

Here is one of the changes I've made waiting till next open to see if orders are triggered.

Code: Select all

if OrderTime == "immidiately" then
        source = ExtSubscribe(1, nil, "t1", instance.parameters.Type == "Bid", "tick");
    else
        source = ExtSubscribe(2, nil, instance.parameters.TF, instance.parameters.Type == "Bid", "open");
    end



Here is an Example of the gap data issue on the backtesting system. I've already turned off the hide gap data feature on marketscope however it does not transfer over to the backtesting system.

I've reached out to FXCM and they've advised that the gap feautre is only there for Aesthetics but as you can see on the image below the backtest is clearly reacting to non-existent price points. They advised they will notify developers to correct issue.

https://drive.google.com/open?id=0B4RA1yOBtVPYVVFIT0w0S0N1VTA

Best
Ron

[Julia CJ]

Hi Ronald,

As you may already know the system is hiding price gaps so when you run a backtest and the strategy opens a position based on the last closed candle it may not actually be how it would play on out on the market

.

Please use Price Gap option:

Also on the same note am trying to figure out how to make the strategy/strategies I currently use to open/start at the beginning of each new H8 or D1 candle based on the current market condition.

According to this question you would be better to contact with author of strategy.

[ronald3rg]

Hi Julia,

Thanks for reply but the Gap data option doesn't work when it comes to backtesting as you can see on image sample.

As for the second questions I have tried everything I could think of just tried creating a function to make strategy basically try again and act on tick values of the open but looks like i coded it wrong.

What I need is for it to act on live candle once for selected TF at the beginning of candle. Right now in order for me to make this one work i have to open change to live and then switch it back off I just needed to do that automatically.

current:

Code: Select all

if Parameters["ACTNOW"] then
       
        Sources["Tick"] = ExtSubscribe(3, nil, "t1", true, "tick");
    else
        Sources["Bid"] = ExtSubscribe(1, nil, instance.parameters.TF, instance.parameters.Type == "Bid", "bar");
    end


[ronald3rg]

This is what I've told it to do in order to remedy situation I'll have to wait till next open session on daily to see if it works unless anyone can point out any errors.


Current:

Code: Select all


    if Parameters["ACTNOW"] then
        --source = ExtSubscribe(1, nil, "t1", instance.parameters.Type == "Bid", "tick");
        Sources["Tick"] = ExtSubscribe(3, nil, "t1", true, "tick");
    else
        Sources["Bid"] = ExtSubscribe(1, nil, instance.parameters.TF, instance.parameters.Type == "Bid", "bar");
     end



On the new code I added a call to new function to try again if we get an open order failed message which currently only happens at market open. If condition is met I call on tryAgain() function which then changes the source to live candle which then calls on restart() function to start again.

I believe since we'd only call on tryAgain() once it should only change the source to tick for the time it is called upon only.

Code: Select all


    if Parameters["ACTNOW"] then
        --source = ExtSubscribe(1, nil, "t1", instance.parameters.Type == "Bid", "tick");
        Sources["Tick"] = ExtSubscribe(3, nil, "t1", true, "tick");
    else
        Sources["Bid"] = ExtSubscribe(1, nil, instance.parameters.TF, instance.parameters.Type == "Bid", "bar");
   

       end

if cookie == 200 and not success then
        terminal:alertMessage(instance.bid:instrument(), instance.bid[instance.bid:size() - 1], "Open order failed" .. message, instance.bid:date(instance.bid:size() - 1));
   
tryAgain();
   
 elseif cookie == 201 and not success then
        terminal:alertMessage(instance.bid:instrument(), instance.bid[instance.bid:size() - 1], "Close order failed" .. message, instance.bid:date(instance.bid:size() - 1));
    end

function tryAgain()
local valuemap = core.valuemap();
Sources["Tick"] = ExtSubscribe(3, nil, "t1", true, "tick");
restart();

end





[Julia CJ]

Hi Ronald,

the Gap data option doesn't work when it comes to backtesting

Please specify which value you set to Price Gaps option.

Concerning the second question, please try to do like this:

Sources["Tick"] = ExtSubscribe(3, nil, "t1", true, "tick");
Sources["Bid"] = ExtSubscribe(1, nil, instance.parameters.TF, instance.parameters.Type == "Bid", "bar");

function ExtUpdate(id, source, period)
if id == 1 then
-- do something on every instance.parameters.TF
else --id == 3!!!
-- do something on live data
end

[ronald3rg]

HiJulia,

Price gaps is set to Show if you take a look at image I added on original post that is a sample of marketscope next to Backtest same time frame. The gaps are not shown when backtesting, If you convert chart to table you'll also notice that the data is completely different for open highs and lows the only number the always matches real data is the close.

this greatly impacts all backtest results so I am just wondering if using another platform like debugger or optimizer will have more accurate price data the ts2 backtester since we can't currently remove the hide gaps feature from the chart.


ON code I already have set up that way just didn't post but it still doesn't open trade/orders on the open this is only an issue during market open, strategy works fine on any other time frame at any other time the issue is specifically isolated to market open.

This issue affects multiple strategies. I've been testing different strategies set to D1 candle and of want positions to open at market open but all of them give me same error message or variation of it. They are all working fine, but it seems that at 0200 when market opens it immediately attempts to execute code and i guess not all systems lineup and order cannot be placed as if market was still closed.

Currently on each single strategy i have to open and switch to live and then switch back or open and close strategy for them to execute at 0200 on daily and repeat each night at 0200. not getting any sleep lol.

[Julia CJ]

Hi Ronald,

Thank you for the useful description. The developers have investigated this problem in Backtester. As a result, such option does not work for strategy data. This defect will be fixed in the upcoming release.

[ronald3rg]

Awesome ,