Optimizer crashes

Section for discussions related to indicators, use of indicators, and building of trading stategies using indicators.

Moderator: admin

Optimizer crashes

Postby TakisGen » Wed Jun 04, 2014 5:42 pm

I have recently developed three indicators which are very promising so i decided to start backtesting and optimization.
One of my problems is that i cannot exclude certain output from the results but that is another story told in another post.
The real problem is that the optimizer is crashing with no obvious reason and 50% of the sessions never end.
Errors that are reported in the crash.txt are of type EXCEPTION_ACCESS_VIOLATION and never occure at the same point.
Today i noticed that in the log there were lines like: "Optimizer could not start strategy: (nil)" and "Critical Error".
So i thought for a while what i was doing that i am not doing it now and what i was not doing that i am doing it now.
I have a habit not letting the processor under heavy load. While developement, all optimization tests were done using only one cpu core. After development i am using three of four.
That was it. Went back using only one core and all optimizations are finishing. No more crashes. (Win7, x64, i5, 4Gb+4 ReadyBoost)
I think that someone should start heavy multithreading stress tests.
Regards
Takis
TakisGen
 
Posts: 34
Joined: Sat May 28, 2011 10:09 am

Re: Optimizer crashes

Postby Apprentice » Thu Jun 05, 2014 7:40 am

Can u share your code.
We need this so that we could learn more about this issue.
User avatar
Apprentice
FXCodeBase: Confirmed User
 
Posts: 36341
Joined: Thu Dec 31, 2009 11:59 am
Location: Zagreb, Croatia

Re: Optimizer crashes

Postby TakisGen » Thu Jun 05, 2014 10:41 am

Hi Apprentice,

I think it is not practical to share a strategy file with 1300 lines of code together with a functions library of 1800 lines, not to mention the relating indicators that are at least modified versions of known ones. The strat file contains some hundred of parameters (it is "all in one") and only i could make it run, so i know it is of no use by someone else exept for educational purposes.

Instead, knowing that the problem is not in the actual logic of one of the strategies or in one of the indicators, i did some further search.

As i stated in my previous post all started trying to exclude passes with dd higher than a certain threshold from the results. Not being able to do that, i decided to flag those passes using the storagedb. Guess what, i was wrong in my previous post when i mentioned that this is another story.
Writting anything to the db causes lines like the following to appear in the log every now and then:

Critical error occurred. Optimization is stopped. 2014-06-05 17:59:10
Optimization pass 319 (10, 5, 7, 1) calculated 2014-06-05 17:59:10
...
Optimization pass 219 (10, 4, 6, 1) calculated 2014-06-05 17:56:32
Optimization pass 218 (9, 4, 6, 1) calculated 2014-06-05 17:56:30
Optimizer could not start strategy. Cause: (nil) 2014-06-05 17:56:28
Optimization pass 217 (7, 4, 6, 1) calculated 2014-06-05 17:56:28
Optimization pass 216 (6, 4, 6, 1) calculated 2014-06-05 17:56:27
...

Please notice that this is a session with 7000 passes never finishing.
You can reproduse the errors by writting anything in the db.
I did a quick test with both :put and pcall(:put) like:
DB.alfa:put("key1","data1")
local ret=pcall(DB.alfa:put("key1","data1 pcall"))
Both are failing now and then when using three of four cores but the same is not true when using one core.
I think that you will be able to reproduce this easily.
Can you please confirm this?

Now i have storagedb removed from the code and investigating if the EXCEPTION_ACCESS_VIOLATION crashes i have mentioned still exist or not. I am not sure if they are related to those errors in the log and dbstorage module. Any thoughts on this?

I will keep you informed about any findings from my side.

Regards
Takis
TakisGen
 
Posts: 34
Joined: Sat May 28, 2011 10:09 am


Return to Discussions

Who is online

Users browsing this forum: No registered users and 9 guests