Optimisation in the Lua Strategy Debugger

Section for all discussions related to development using Indicore API/SDK.

Moderator: admin

Optimisation in the Lua Strategy Debugger

Postby Exolon » Sun Jul 10, 2011 9:44 am

Hi,

When using genetic optimisation of strategy parameters, it's very often the case that a quite good candidate (or even the best one) is found early in the search. Since backtesting with large datasets (e.g. the 6 month m1 EURUSD prices) takes a long time, it's a waste of time if we find an excellent solution in pass 25 but keep evaluating 2000 more solutions (maybe an hour!).

Could I suggest that during the search, the quality of the best solution so far (and the number of the pass when it was found) be shown in either the search alert box or the "OptimiseResults" tab?
Then we could quickly cancel the search when we see a "pretty good" solution? This is especially useful when doing very rough exploration of parameters and e.g. timeframes.

Also, strategy time frame can't be optimised - this seems like a critical variable whose appropriate setting is not always obvious. Could it be added to the optimisable variables?
Exolon
 
Posts: 31
Joined: Thu Jul 07, 2011 5:39 pm
Location: Ireland

Re: Optimisation in the Lua Strategy Debugger

Postby sunshine » Mon Jul 11, 2011 8:23 am

The beta version of the platform will be published on this site pretty soon. It will support strategy parameter optimization. You will be able to view results during the optimization process and stop optimization at any time.

Please watch for information about the releases on this site.

As for the strategy time frame optimization, I think it's a good idea. I'll pass the suggestion forward for future consideration. Thank you.
sunshine
 

Re: Optimisation in the Lua Strategy Debugger

Postby Exolon » Thu Jul 14, 2011 4:44 pm

When optimising a strategy with lots of parameters, it's quite a pain to get the "There are too many combinations (23592960) for optimization. Optimization is not aplicable. Maximum is 10000000 combinations" message.

Could I make one more request and ask that when optimising strategy parameters, at least using GAs, the "too many combinations" check be dropped? It might be logical to say "this is computationally infeasible using a brute-force search", but it doesn't really make sense for a heuristic, self-guiding search strategy like GA, which is much better at weeding out the low-quality settings than humans, even with a huge number of dimensions (GAs are routinely used to optimise solutions with a search space of hundreds of continuous numbers).
Exolon
 
Posts: 31
Joined: Thu Jul 07, 2011 5:39 pm
Location: Ireland

Re: Optimisation in the Lua Strategy Debugger

Postby sunshine » Fri Jul 15, 2011 8:15 am

It's not a problem to remove this limitation.
But we suppose that so big number of combinations (you are trying to use about 30 mln combinations) couldn't guarantee accuracy of optimization results.
This is for a few reasons:
1) The bigger is the number of combinations, the higher probability of curve fitting.
2) The usage of so big number of combinations will cause dropping of very big number of combinations. And even usage of GA approach couldn't guarantee that the best combinations will be dropped. This makes optimization pretty useless.
Such optimization is also pretty useless due to time and performance efforts.
sunshine
 

Re: Optimisation in the Lua Strategy Debugger

Postby Exolon » Fri Jul 15, 2011 2:27 pm

sunshine wrote:It's not a problem to remove this limitation.
But we suppose that so big number of combinations (you are trying to use about 30 mln combinations) couldn't guarantee accuracy of optimization results.
This is for a few reasons:
1) The bigger is the number of combinations, the higher probability of curve fitting.
2) The usage of so big number of combinations will cause dropping of very big number of combinations. And even usage of GA approach couldn't guarantee that the best combinations will be dropped. This makes optimization pretty useless.
Such optimization is also pretty useless due to time and performance efforts.


Hi sunshine,

With any heuristic search method that avoids an exhaustive search, we cannot guarantee that optimal results will be discovered. But as is suggested on the site, we might settle for "very good" results rather than optimal.
The optimisation certainly will not be useless; for example, if you start a run and abort after only 2 or 3 generations, there are nearly always some excellent candidate solutions already found.
In many cases, the search space is fairly smooth and so optimisation can at least help automatically guide us to insights about good and bad ranges for most parameters (e.g. a parameter could range from [1-100], but a few generations of optimisation might quickly narrow it down to the range [40-50]).

Look at it this way: even if it was pretty useless, it couldn't be worse than the current behaviour which does not allow you to even attempt the optimisation. It still must be better than being forced to manually explore and backtest with the parameters, right?
Exolon
 
Posts: 31
Joined: Thu Jul 07, 2011 5:39 pm
Location: Ireland

Re: Optimisation in the Lua Strategy Debugger

Postby Andrio » Sat Dec 20, 2014 8:21 am

Then I might run another optimisation session (with parameters in a narrower range based on the results of the first optimisation) on the big file and then backtest on the small file.
FSL:
Andrio
 
Posts: 1
Joined: Sat Dec 20, 2014 8:17 am


Return to Discussions

Who is online

Users browsing this forum: No registered users and 3 guests