## Optimizing averages.lua: Optimization by example

### Optimizing averages.lua: Optimization by example

The old implementation of averages.lua (viewtopic.php?f=17&t=2430) is a good example on how to do not write the indicator to have it fast and efficient. Every rule of the specified in the http://fxcodebase.com/wiki/index.php/In ... timization article is broken. So, let see the highlight which helped to make it 2-6 times faster.

Generic indicator structure optimization:

1) String comparison in Update() function. You know that Update() is called every time when a new tick comes to the live market snapshot. So, we can recognize the Update function as called inside a loop. Now, the whole Update() function is just a big sequence of string comparisons to choose the proper function for the Method chosen during initialization. But the method does NOT change. Why do we need make that selection all the time? The better way is to store the proper function once, in the Prepare() method into the global variable and then call it.

was:
Code: Select all
`function Update(period, mode) local i;   if (period>=first) then    if Method=="MVA" then     MA[period]=MVA(source,period,MA,Period);    elseif Method=="EMA" then     MA[period]=EMA(source,period,MA,Period);...`

now:
Code: Select all
`local UpdateFunction;function Prepare(onlyName)    if Method=="MVA" then     UpdateFunction=_G["MVA"];    elseif Method=="EMA" then     UpdateFunction=_G["EMA"];...endfunction Update(period, mode) local i;   if (period>=first) then     UpdateFunction(source,period,MA,Period);...`

2) Needless internal streams. The indicator always creates five internal stream, even these streams aren't required at all. Handling of every, even unused, stream takes time on every tick. So, just create the streams ONLY in case they are actually required.

Algorithm optimization:

1) A lot of algorithms, such as simple moving average, linear-weighted moving average, least-square moving average are implemented in core in the mathex table. This table is implemented in C++, so it is faster than Lua.

Was:
Code: Select all
`  local Weight=0;  local Sum=0;  local i;  for i=0,Period_-1,1 do   Weight=Weight+(Period_-i);   Sum=Sum+Price[shift-i]*(Period_-i);  end  if Weight>0 then   return Sum/Weight;  else   return 0;  end else  return 0; end`

now
Code: Select all
`  return mathex.lwma(Price, Period - Period_ + 1, Period);`

Also, it is a good idea to understand the formula beneath the somebody's source first, because a part of the formula may be optimized. For example, for ILRS the slope may be calculate using mathex:

was:
Code: Select all
`function ILRS(Price,shift,MAarray,Period_) local sum=Period_*(Period_-1)*0.5; local sum2=(Period_-1)*Period_*(2*Period_-1)/6; local sum1=0; local sumy=0; local i; for i=0,Period_-1,1 do  sum1=sum1+i*Price[shift-i];  sumy=sumy+Price[shift-i]; end local num1=Period_*sum1-sum*sumy; local num2=sum*sum-Period_*sum2; local slope; if num2~=0 then  slope=num1/num2; else  slope=0; end return slope+MVA(Price,shift,MAarray,Period_);end`

now:
Code: Select all
`params.buffer[period] = mathex.lregSlope(params.source, from, period) + mathex.avg(params.source, from, period);`

2) Needless calculation

Even if algorithm is not implemented in full or partially in the mathex, there is no need to do repeating calculations which have exactly the same result for every tick. For example weights for SineWMA are the same and depends on the offset to the last bar only.

Code: Select all
`function SineWMA(Price,shift,MAarray,Period_) if shift>=first then  local Sum=0;  local Weight=0;  local i;  r=core.range(math.max(shift-Period_+1,first),shift);  for i=r.from,shift,1 do   Weight=Weight+math.sin(math.pi*(shift-i+1)/(Period_+1));   Sum=Sum+Price[i]*math.sin(math.pi*(shift-i+1)/(Period_+1));  end  if Weight>0 then   return Sum/Weight;  else   return 0;  end endend`

now
Code: Select all
`---- SineWMA: Sine weighted moving average--function SineWMAInit(source, n)    local p = {};    p.source = source;    p.n = n;    p.offset = n - 1;    p.sine = {};    p.first = source:first() + n - 1;    local i, w;    w = 0;    for i = 1, n, 1 do        p.sine[i] = math.sin(math.pi * (n - i + 1) / (n + 1));        w = w + p.sine[i];    end    p.weight = w;    p.alwaysZero = (w == 0);    return p;endfunction SineWMAUpdate(params, period, mode)    local sum = 0;    if not params.alwaysZero then        local src = params.source;        local sine = params.sine;        local n = params.n;        local p = period - n;        for i = 1, n, 1 do            sum = sum + src[p + i] * sine[i];        end        sum = sum / params.weight;    end    params.buffer[period] = sum;end`

So, as you can see all things are pretty simple, but provides a good performance improvements. When we want to use our indicators in 1-year long backtesting, and, especially, optimization, every additional second counts. 6 times optimization of the indicator may make 10-minute backtesting just a 1.5 minute long. A few hours of indicator optimization worths it, right?
Nikolay.Gekht
FXCodeBase: Base Builder

Posts: 1223
Joined: Wed Dec 16, 2009 6:39 pm
Location: Cary, NC

### Re: Optimizing averages.lua: Optimization by example

Hello Nikolay,

1.I wanted to ask you if there is in any way you can explain to me how there is a way we can actually use C+ database. I.E. for the optimizations to the algorithms implemented you said used C+ calculation in order to improve the speed due to the fact C+ is faster. Is there an index of operations you can share which you find using C+ functions and run smoother?

2. This is about the string values.

Code: Select all
`function VAMAInit(source, n)function ARSIInit(source, n)`

What is the purpose of "n"?
cnikitopoulos94

Posts: 50
Joined: Sat Sep 12, 2015 8:10 am

### Re: Optimizing averages.lua: Optimization by example

cnikitopoulos94,

You can see the description of mathex and all of its methods here:
http://www.fxcodebase.com/documents/IndicoreSDK/mathex.html
Georgiy
FXCodeBase: Initiate

Posts: 151
Joined: Tue Jul 29, 2014 4:49 am