# Why Other Time Frame Data is Required?

Sometimes an indicator requires data of a bigger time frame. For example, you could use the close of the previous day for calculations. However, it’s possible to try and find this data in the price history the indicator is applied to. To get yesterday’s high/low, the whole yesterday’s data must be loaded. If the current history is a minute history, in the worst case, you would need to have at least 2880 bars to be able find that data (1440 minute bars for today and then 1440 minute bars for the whole yesterday). This is a huge amount to load and then to calculate but the system has access to the day data, so why don’t you use it? You can do it.

However, there are a number of points you should handle, carefully. You can develop an indicator which shows yesterday’s close price on the chart and discuss each point in detail.

# What is “Yesterday”?

The handling of the date in indicators is quite simple. A date value is a number, which is just a number of days past midnight of Dec, 30 1899. The format is also known as Windows Ole Automation date format. The integer part of this number is a number of days and the fractional part is a part of the day. So, to find a calendar midnight date and time, you should just floor the value (cut away the fractional part). To shift the date to N days ago, you should just subtract N from the date. So, if FXCM used the “calendar clock” for the trading days (i.e. the trading day starts at midnight), the finding of yesterday for bar x of the source would be pretty simple: `math.floor(source:date(x)) - 1`.

Unfortunately, FXCM’s trading day does not match the calendar trading day. Currently, the trading day starts at 17:00EST yesterday and lasts until 17:00EST today. For different configurations, this date can differ. Hopefully, the indicator core helps you to handle calculation of the candle borders for a particular date. There is a `core.getcandle` function which returns the date and time of the candle of the specified time frame to which the specified date and time belong. As you can see, this function requires two parameters which let you “tune” this function for a certain configuration: an offset of the trading day and an offset of the trading week against calendar. You should use `host:execute("getTradingDayOffset")` and `host:execute("getTradingWeekOffset") ` host’s calls to get the settings of the trading day and trading week for particular host application and connection.

So, the function to get yesterday’s date properly can look as follows:

```function getYesterday(date)
local offset;
local weekoffset;

local today, yesterday;
today = core.getcandle("D1", date, offset, weekoffset);
yesterday = today - 1;
return yesterday;
end```

And yet, there is another point you should handle. There are “non-trading” periods when neither trading is possible nor prices are received. So, during a week, only Monday trough Friday candles exist. There are no “Saturday” candles at all. So, when your minute belongs to a Monday day candle, the function returns, to you, a Sunday candle which just does not exist. So, for Monday’s data you should take the previous Friday’s close price.

To handle that correctly, there is another useful function: `core.isnontrading` which checks whether the date belongs to a non-trading period (Friday 17:00EST-Sunday 17:00EST) and returns the date and time when this non-trading period is finished. So, to find the proper “yesterday” candle you have to check whether the calendar yesterday falls into non-trading period and shift the “yesterday” date into past to skip this non-trading period.

You can modify your function, a bit:

```function getYesterday(date)
local offset;
local weekoffset;

local today, yesterday;
today = core.getcandle("D1", date, offset, weekoffset);
yesterday = today - 1;

return yesterday;
end```

So, now, having the `getYesterday()` function, you can easily find how many days you need. The oldest day is `getYesterday(source:date(source:first()))`, i.e. the “yesterday” bar which corresponds to the oldest available bar. And what about the newest one? There are two cases. The source collection could be either a historical (i.e. a collection loaded until the specified date which does not change for the chart window) or “alive” (i.e. loaded till “now” and continuously growing as new bars come). According to these two cases, the “to” date could be either a “yesterday” day of the latest available bar (`getYesterday(source:date(source:size() - 1))` or … now as well. To specify “now”, you use `0` value.

## Using Host to Load Data

Ok, now you know which data you need. The next step is to get this data. Marketscope provides a `host` interface which lets you execute an application-specific functions. One of these functions is `“getHistory”`. By this call Marketscope loads the specified instrument in the specified time frame and in the specified time range. See `host:execute(“getHistory”, …) ` for details.

```local days;     -- a variable to keep days

-- the function loads the proper day history
if source:size() < source:first() then
return ;
end

local from, to;
-- load the data from the date of proper yesterday for
-- the first available bar of the source
from = getYesterday(source:date(source:first()));
-- load to "now" if the source is up to "now" or
-- to the day which corresponds to the latest
-- day of the collection.
if source:isAlive() then
to = 0;
else
to = getYesterday(source:date(source:size() – 1));
end
days = host:execute("getHistory", 1, source:instrument(), "D1", from, to, source:isBid());
end```

It is important that after the `“getHistory”` call, Marketscope only starts to load the data. It takes some time but Marketscope does not wait until the data is completely loaded to avoid hanging of the indicator. When data loading is finished, Marketscope notifies the indicator by calling the `AsyncOperationFinished` function. Moreover, Marketscope is not affected by the indicator status and continues calling indicator to be updated.

So, there are two things to do.

## Protect Indicator from Updating While the Data Is Loaded

First, you should prevent the indicator from updating while the data is loaded.

To prevent the indicator from being recalculated, you can return from the update function while data is being loaded. Add the “loading” global variable so that all functions could “know” that data is being loaded.

```local days;         -- day source data

function Prepare()
...
days = nil;
...
end

function Update(period, mode)
-- do nothing if data is being loaded
return ;
end

if days == nil then
return ;
end
...
end

assert(success, error);
end
return 0;
end

return ;
end

local from, to;

-- load the data from the date of proper yesterday for
-- the first available bar of the source
from = getYesterday(source:date(source:first()));
-- load to "now" if the source is up to "now" or
-- to the day which corresponds to the latest
-- day of the collection.
if source:isAlive() then
to = 0;
else
to = getYesterday(source:date[source:size() - 1]);
end
days = host:execute("getHistory", 1, source:instrument(), "D1", from, to, source:isBid());
end```

## Forcing Update When Data Is Loaded

Now, you need to make the indicator update when data is loaded. There is function `instance:updateFrom()` which 'forces' the indicator to be recalculated from the specified period. Also, when `AsyncOperationFinished` returns the `core.ASYNC_REDRAW` value, Marketscope redraws the indicator in the chart window.

So, since you haven’t calculated anything yet, you have to update your indicator right from the oldest data. Just fix `AsyncOperationFinished` a bit.

```function AsyncOperationFinished(cookie, success, error)
assert(success, error);
instance:updateFrom(source:first());
return core.ASYNC_REDRAW;
end
return 0;
end```

## Getting Data

It is almost finished. The last question is how to get the data. You can calculate the day date. Of course, you can just loop through the day history and find the date but it is a bit long. There is a function, however, that finds the date just for log2(n) operations - `core.findDate()`.

So, you can build the first version of your indicator. You can do so and try it.

```function Init()
indicator:name("Show Yesterday Close");
indicator:description("The indicator shows yesterday's close value on the chart");
indicator:requiredSource(core.Tick);
indicator:type(core.Indicator);

indicator.parameters:addColor("clr", "Indicator Line Color", "", core.rgb(255, 255, 0));
end

local source;       -- indicator source data
local first;        -- first available source data
local days;         -- day source data
local SYC;          -- yesterday close output

local host;
local offset;
local weekoffset;

function Prepare()
local name;

name = profile:id() .. "(" .. instance.source:name() .. ")";
instance:name(name);

source = instance.source;
first = source:first();
days = nil;

host = core.host;

SYC = instance:addStream("SYC", core.Line, name, "SYC", instance.parameters.clr, first);
end

function Update(period, mode)
if period < first then
return ;
end
return ;
end

-- load the data if there is no day
if days == nil then
SYC:setBookmark(1, source:first());
return ;
end

-- try to find the value in the days collection
local yesterday;
yesterday = getYesterday(source:date(period));
SYC:setBookmark(1, period);
return ;
end

local i;
i = core.findDate(days, yesterday, false);
if i >= 0 then
SYC[period] = days.close[i];
end
end

assert(success, error);
local i = SYC:getBookmark(1);
if i < source:first() then
i = source:first();
end
instance:updateFrom(i);
return core.ASYNC_REDRAW;
end
return 0;
end

-- the function loads the proper day history
if source:size() < source:first() then
return ;
end

return ;
end

local from, to;

if days == nil then

-- load the data from the date of proper yesterday for
-- the first available bar of the source
from = getYesterday(source:date(source:first()));
-- load to "now" if the source is up to "now" or
-- to the day which corresponds to the latest
-- day of the collection.
if source:isAlive() then
to = 0;
else
to = getYesterday(source:date(source:size() - 1));
end
days = host:execute("getHistory", 1, source:instrument(), "D1", from, to, source:isBid());
else
from = getYesterday(source:date(source:first()));
to = days:date(0);
host:execute("extendHistory", 1, days, from, to);
end
end

-- returns the beginning of the "yesterday" date for
-- the specified date
function getYesterday(date)
local today, yesterday;

-- get beginning of the today's candle and round it up to the second
today = core.getcandle("D1", date, offset, weekoffset);
today = math.floor(today * 86400 + 0.5) / 86400;

-- get calendar yesterday
yesterday = today - 1;

-- if the calendar yesterday date is non-trading (Friday 17:00 EST - Sunday 17:00 EST)
-- shift three days back against the non-trading period end (Sunday 17:00 EST).
end
return yesterday;
end```

Now apply the indicator to the chart.

It works, so you can stop. When you try to load more data into past it apparently stopped to work.

## How To Process the Scroll Back

What happened? Well, remember that you loaded the data starting from yesterday for the oldest of the existing candles. When you started to scroll your chart back, more data was loaded. No one updated your collection of day bars. So, you have to force Marketscope to load new data when the source collection is extended into past. Good news is that there is another cool call of the host interface: `host:execute(“extendHistory”)` which helps you just load more data into already loaded history instead of loading the whole history again. Since the original source history can be extended only into past, it is too easy to check whether new data is loaded. When it happens, the “yesterday” value you are trying to find is earlier than the date that you previously loaded the day history from.

So, do the following modification in your code:

1. Keep the “from” date that you loaded the day history earlier.

So, add new `loadedFrom` global variable and change the `Update` and `loadData` functions.

There is a new indicator version:

```function Init()
indicator:name("Show Yesterday Close");
indicator:description("The indicator shows yesterday's close value on the chart");
indicator:requiredSource(core.Tick);
indicator:type(core.Indicator);

indicator.parameters:addColor("clr", "Indicator Line Color", "", core.rgb(255, 255, 0));
end

local source;       -- indicator source data
local first;        -- first available source data
local days;         -- day source data
local SYC;          -- yesterday close output

local host;
local offset;
local weekoffset;

function Prepare()
local name;

name = profile:id() .. "(" .. instance.source:name() .. ")";
instance:name(name);

source = instance.source;
first = source:first();
days = nil;

host = core.host;
SYC = instance:addStream("SYC", core.Line, name, "SYC", instance.parameters.clr, first);
end

function Update(period, mode)
if period < first then
return ;
end
return ;
end

-- load the data if there is no day
if days == nil then
return ;
end

-- try to find the value in the days collection
local yesterday;
yesterday = getYesterday(source:date(period));

-- check whether such day bar is not loaded yet
return ;
end

-- find the day data
local i;
i = core.findDate(days, yesterday, false);
if i >= 0 then
SYC[period] = days.close[i];
end
end

assert(success, error);
instance:updateFrom(source:first());
return core.ASYNC_REDRAW;
end
return 0;
end

-- the function loads the proper day history
if source:size() < source:first() then
return ;
end

return ;
end

local from, to;

if days == nil then

-- load the data from the date of proper yesterday for
-- the first available bar of the source
from = getYesterday(source:date(source:first()));
-- load to "now" if the source is up to "now" or
-- to the day which corresponds to the latest
-- day of the collection.
if source:isAlive() then
to = 0;
else
to = getYesterday(source:date(source:size() - 1));
end
days = host:execute("getHistory", 1, source:instrument(), "D1", from, to, source:isBid());
else
from = getYesterday(source:date(source:first()));
to = days:date(0);
host:execute("extendHistory", 1, days, from, to);
end
end

-- returns the beginning of the "yesterday" date for
-- the specified date
function getYesterday(date)
local today, yesterday;

-- get beginning of the today's candle and round it up to the second
today = core.getcandle("D1", date, offset, weekoffset);
today = math.floor(today * 86400 + 0.5) / 86400;

-- get calendar yesterday
yesterday = today - 1;

-- if the calendar yesterday date is non-trading (Friday 17:00 EST - Sunday 17:00 EST)
-- shift three days back against the non-trading period end (Sunday 17:00 EST).
end
return yesterday;
end```

Install the indicator and try it. Now, when the previous version stops working during the scroll, the new version works. Congratulations!

# Notes

## How to Debug Such Indicators Using SDK Debugger?

First, you must prepare the data. You can save price data files using Marketscope. Just open some instrument in both resolutions (for example 30 minutes and 1 day) and then save both histories in the Indicore SDK data format (File->Export to Indicore…). You can also download the data automatically via SDK Debugger using the quotes manager server (Tools->Load Quotes).

When you start the indicator, choose the smaller timeframe history in the indicator parameters. Please, also check the “Predeliver Data” checkbox in the indicator parameters to let the debugger show the whole available history to the indicator, like Marketscope does. Without this option, the debugger shows the history only up to the currently simulated bar. When the indicator executes `“getHistory”` call, the prompt for data appears. Choose the day data previously saved from Marketscope, in that prompt, or use the data from the quotes manager server.

## How to Handle Timeframes Other than 1 Day?

To make the example indicator simpler, I used a 1-day time frame. However, you would want to do the same for other time frame data, for example, use 1-hour close price instead of a day close price. To do that, you can:

• Let the user choose the time frame. Add a string parameter to the indicators parameter list and then set the time frame selector flag to that parameter. As a result, in the Prepare function, the corresponding parameter contains the code of the time frame chosen by the user.
```function Init()
...
indicator.parameters:setFlag("TF", core.FLAG_PERIODS);
...
end```
• Calculate the length of a candle of the specified time frame in day. Use the [c]core.getcandle()[/c] function to get the start date of any candle and the start date of the next candle. The difference between these two values is the length of the candle in days. It is also strongly recommended that you calculate all of the dates in seconds, rather than in days, to avoid rounding errors.
```local candleLength;  -- length of candle in seconds

function Prepare()
...
...
local thisCandle, nextCandle;
thisCandle, nextChandle = core.getcandle(instance.parameters.TF, core.now(), offset, weekoffset);
candleLength = math.floor((nextCandle - thisCandle) * 86400 + 0.5);
...
end```
• Use previously calculated candle length instead of 1 day value in your “getPreviousBar” function. Please note that while the candle length could be different, the non-trading period is still in days. So, handle it carefully.
```function getPreviousBar(date)
local curr, prev;

-- get beginning of the today's candle and round it up to the second
curr = core.getcandle(instance.parameters.TF, date, offset, weekoffset);
today = math.floor(today * 86400 + 0.5);

-- get calendar yesterday
prev = (curr – candleLength) / 86400;

prev = prev - 2;
prev = math.floor(prev * 86400 + 0.5);

-- get calendar yesterday
prev = (prev – candleLength) / 86400;
end
return prev;
end```