Crash when optimising Logarithmic regression strategy

[Exolon]

Hi,

When I run a GA optimisation on the Logarithmic regression strategy using LuaJIT in JIT mode, it crashes after a while (perhaps with certain "evil" parameters).

Windows produced the following almost useless crash report:

Problem signature:
Problem Event Name: APPCRASH
Application Name: luadbg1.exe
Application Version: 0.0.0.0
Application Timestamp: 4e036796
Fault Module Name: Lua5.1.jit.dll
Fault Module Version: 0.0.0.0
Fault Module Timestamp: 4e03672e
Exception Code: c0000005
Exception Offset: 0000fd8b
OS Version: 6.1.7601.2.1.0.768.3
Locale ID: 6153
Additional Information 1: 8384
Additional Information 2: 83844aaa1396ba020f3f55792bf9609e
Additional Information 3: 1b23
Additional Information 4: 1b239fd70b497763884439a08a4a481a

I recompiled the latest LuaJIT and tried to replace the DLL, but it crashed immediately on launching the strategy debugger; maybe it needs VS rather than MinGW. Also I'm running 64-bit Windows, if this makes a difference.

Will retest with vanilla Lua if there's enough time...

[sunshine]

Thanks for the report.
I've forwarded it to Development.

[Exolon]

If it helps, optimising "MomentumTradeStrategy" crashes immediately for me if I do this:

• Open optimisation dialog
• Select MomentumTradeStrategy
• Set the first instrument price file
• Expand "Price" in the parameters section and select "median"
• Start the run

It may not be related though, as the error message is slightly different (a runtime C++ error in this case, trying to call a pure virtual function that was presumably supposed to be implemented in a subclass).

Unfortunately the crasher with logarithmic regression strategy (and polynomial, probably because it seems to use the logarithmic indicator) take a long time to replicate; I'm not sure what the conditions are.

[sunshine]

I've noticed that the Debugger crashes if a strategy uses a custom indicator.

For example, Logarithmic regression strategy uses the Logarithmic regression indicator.
So to be able to backtest and optimize the strategy, you should copy the indicator to the Indicators folder (by default the path is C:\Gehtsoft\IndicoreSDK\indicators).
This solved the issue for me.
Usually indicators required for a strategy are listed in the post with the strategy.

Please let me know if this won't help.

[Exolon]

Hi,

\Gehtsoft\IndicoreSDK\indicators contains Logarithmic_Regression.lua and Polynomial_Regression.lua. The strategies work fine usually for a few thousand optimisation passes.

I left the machine optimising Logarithmic strategy parameters overnight with the normal Lua VM and found that it had not crashed. But then when I did a backtest, it stayed at 0 ticks forever until I killed the program. So it doesn't seem like a LuaJIT problem.

Perhaps an error in the FFI code, or a race condition in dealing with the multiple agents?

[sunshine]

Could you please specify parameteres you are using for optimization?

[Exolon]

First instrument price file: C:\Gehtsoft\IndicoreSDK\data\a-EURUSD-m1-6M-2011.csv
Is hedging allowed? True
Initial amount: 2000
MMR: 50
Method: Genetic
Criteria: Highest balance
Agents number: 4 (it also happens with "Default"... I might leave it overnight with #agents = 1 to see if it could be a problem with the OpenMP code)
Period: min 49 max 60 step 1
Deviation: 3
Price: low
Distance: 5 ; 10 ; 1
Type of signal: reverse
Time frame: m15
Allow strategy to trade: True
Account to trade one: TESTACCT
Trade amount: 1;1;1
Set limit orders: True
Limit order in pips: 23; 28; 1
Set stop orders: True
Stop order in pips: 14; 30 ; 1
Trailing stop order: False; True
Show alert (and following parameters): False

(crashed on pass 143)

Wow, I just realised that stop orders do not function if "set stop orders" is false, even if "trailing stop order" is true. Perhaps a little counterintuitive... maybe a warning might be in order if the options are set like this?

As I suggested above, it might just be a race condition or something in the OMP code, and maybe only a coincidence that it keeps happening in the Log/Polynomial strategy test because I've spent more time examining them (because their prediction value seems very good!).

[Exolon]

I might know why this is happening. Another crash when optimising the "Repulse Divergence" strategy, but this one printed a bunch of error messages stating that it had run out of memory. At the time of the crash, the Strategy Debugger was taking up 1.2 GB of the 4 GB RAM in this machine.

It turns out that the amount of memory used during an optimisation run grows linearly. I restarted the Strategy Debugger and started the optimisation run again, watching the "private bytes history" graph in Process Explorer. It grew in a straight line as the run progressed, and in 8 minutes 12 seconds increased from 100MB to 600MB. This is almost exactly 1 MB per second.

I noticed a "save log" option in the File menu. This produced a ~90MB logfile containing simulated tick/account history, but only for the very last evaluation/backtest. Doing another run on a small datafile and saving again produced a small log, but the RAM usage didn't drop.

Could it be the case that the data structures holding the log information are not being freed when a new log is started?

Because of this, it is impossible to carry on optimising beyond a fixed number of evaluations, since:
1) LuaJIT has a maximum heap size of ~2GB (not sure about vanilla Lua)
2) Computers have a fixed amount of memory, and even if Lua has a larger addressable heap, it may refuse to allocate from virtual memory.

On the plus side, at least I can now step in and halt an optimisation run if the RAM usage goes over a gigabyte...
Sorry this was so long, but hopefully it helps.

[sunshine]

Thanks for the info. This is very helpful.
Passed to developers. I'll keep you informed.

[Exolon]

Edit: Disregard, I was wrong! The slower memory leakage was because I'd set the optimisation to single thread instead of 3 or 4 threads. So the cause is still a mystery. Sorry if my post was misleading...

Hi guys,

FYI I found a workaround which helps in the meantime. It seems that all the strategy instances are loaded into the global namespace with loadfile(). I'm not an expert on these matters, but maybe you can work around this with some metatable magic, or maybe the functions generated by loadfile() are being stored in some persistent table and hence not being garbage collected.

Marking the global namespace as nil (!) from the strategy's cleanup method seems to work for me so far. Memory usage is still increasing, but much more slowly (about 1MB every minute now):

Code: Select all

function ReleaseInstance()
   _G = nil
end