Live chart data export to CSV

Here you can post and download custom indicators. PLEASE: Do not start topics unless you are posting your own indicator, they will be moved to appropriate section even if you do.

Moderator: admin

Re: Live chart data export to CSV

Postby Gennadiy » Mon Jan 06, 2014 3:06 am

Hello chimpy,

Would it be possible to add a function so that the script can output bars of any minute length as we are currently its reststricted to 1, 5, 15, 30, 60 minute bars.

Instead of changing the current stable regime of <TimeFrame>m5</TimeFrame> and <TimeFrame>H4</TimeFrame> perhaps something simple that added the 1minute bars up and was controlled by say: <TimeFrame>mm125</TimeFrame> <TimeFrame>mm332</TimeFrame>

Although, I guess someone will come along and ask for 9 Hour bars in the future... would this cause a large script overhead adding up 540 minutes?


Unfortunately, the ForexConnect API does not allow to implement this function.
Gennadiy
 

Re: Live chart data export to CSV

Postby chimpy » Fri Jan 17, 2014 8:56 am

Gennadiy ,

Here are the settings I am using and some sample data the app is pulling down.
The data is only running through to Thursday night instead of friday, then starting again on Sunday.
This is not consistent with the marketscope data. I assume its the same on other timeframes and symbols too but have not extensively checked it yet. can you please have a look

Thanks

<URL>www.fxcorporate.com/Hosts.jsp</URL>
<Connection>Real</Connection>
<SessionID></SessionID>
<Pin></Pin>
<TradingDayOffset>-7</TradingDayOffset>
<TradingWeekOffset>0</TradingWeekOffset>
<Delimiter>,</Delimiter>
<NeedLastCompletedCandle>N</NeedLastCompletedCandle>
<TimeZone>UTC</TimeZone>
<SeparateDateAndTime>Y</SeparateDateAndTime>
<FormatDecimalPlaces>Y</FormatDecimalPlaces>

Code: Select all
31/10/2013,21:00,1.60359,1.60453,1.59076,1.59189,1.60387,1.60478,1.59102,1.59286,66594
03/11/2013,22:00,1.59243,1.59773,1.59025,1.59669,1.59282,1.59794,1.59048,1.59702,46844
04/11/2013,22:00,1.59669,1.60616,1.59473,1.60415,1.59702,1.60639,1.59498,1.60447,63592
05/11/2013,22:00,1.60415,1.61168,1.60408,1.60779,1.60447,1.61191,1.60430,1.60807,54797
06/11/2013,22:00,1.60779,1.61130,1.60091,1.60939,1.60807,1.61155,1.60110,1.60969,91867
07/11/2013,22:00,1.60939,1.61044,1.59563,1.60157,1.60969,1.61065,1.59586,1.60192,81912
10/11/2013,22:00,1.59997,1.60209,1.59646,1.59868,1.60042,1.60233,1.59670,1.59900,46371
11/11/2013,22:00,1.59868,1.59914,1.58533,1.59018,1.59900,1.59932,1.58555,1.59062,69155
12/11/2013,22:00,1.59018,1.60666,1.58776,1.60546,1.59062,1.60687,1.58797,1.60588,73004
13/11/2013,22:00,1.60546,1.61003,1.59874,1.60631,1.60588,1.61026,1.59900,1.60677,75846
14/11/2013,22:00,1.60631,1.61342,1.60472,1.61140,1.60677,1.61366,1.60495,1.61217,57005
17/11/2013,22:00,1.61168,1.61477,1.60803,1.61083,1.61301,1.61499,1.60831,1.61115,53921
18/11/2013,22:00,1.61083,1.61313,1.60585,1.61181,1.61115,1.61336,1.60608,1.61216,65643
19/11/2013,22:00,1.61181,1.61770,1.60862,1.61028,1.61216,1.61793,1.60890,1.61093,85818
20/11/2013,22:00,1.61028,1.61996,1.60714,1.61967,1.61093,1.62034,1.60740,1.62019,77659
21/11/2013,22:00,1.61967,1.62236,1.61751,1.62208,1.62019,1.62277,1.61779,1.62274,58404
24/11/2013,22:00,1.62203,1.62391,1.61325,1.61536,1.62297,1.62415,1.61345,1.61568,53981
25/11/2013,22:00,1.61536,1.62180,1.61378,1.62120,1.61568,1.62204,1.61400,1.62183,55458
26/11/2013,22:00,1.62120,1.63296,1.61966,1.62848,1.62183,1.63322,1.61989,1.62888,67296
27/11/2013,22:00,1.62848,1.63565,1.62763,1.63392,1.62888,1.63585,1.62783,1.63487,47340
28/11/2013,22:00,1.63392,1.63835,1.63132,1.63612,1.63487,1.63856,1.63157,1.63755,64531
01/12/2013,22:00,1.63645,1.64416,1.63417,1.63532,1.63768,1.64446,1.63441,1.63585,131212
02/12/2013,22:00,1.63532,1.64359,1.63454,1.63885,1.63585,1.64380,1.63475,1.63927,116136
03/12/2013,22:00,1.63885,1.64029,1.63222,1.63810,1.63927,1.64072,1.63284,1.63862,163331
04/12/2013,22:00,1.63810,1.64028,1.62996,1.63323,1.63862,1.64048,1.63017,1.63363,172992
05/12/2013,22:00,1.63323,1.63922,1.62909,1.63428,1.63363,1.63949,1.62940,1.63527,147259
08/12/2013,22:00,1.63431,1.64315,1.63218,1.64270,1.63533,1.64340,1.63242,1.64305,90710
09/12/2013,22:00,1.64270,1.64650,1.64178,1.64435,1.64305,1.64686,1.64199,1.64474,97668
10/12/2013,22:00,1.64435,1.64568,1.63381,1.63738,1.64474,1.64590,1.63405,1.63783,114016
11/12/2013,22:00,1.63738,1.64171,1.63201,1.63469,1.63783,1.64193,1.63219,1.63539,120544
12/12/2013,22:00,1.63469,1.63591,1.62615,1.62925,1.63539,1.63619,1.62639,1.63052,109957
15/12/2013,22:00,1.62878,1.63480,1.62833,1.62984,1.63026,1.63504,1.62905,1.62999,104193
16/12/2013,22:00,1.62984,1.63350,1.62144,1.62618,1.62999,1.63372,1.62183,1.62663,108314
17/12/2013,22:00,1.62618,1.64843,1.62603,1.63879,1.62663,1.64861,1.62637,1.63932,176764
18/12/2013,22:00,1.63879,1.63959,1.63346,1.63713,1.63932,1.63997,1.63366,1.63750,110981
19/12/2013,22:00,1.63713,1.63938,1.63145,1.63259,1.63750,1.63963,1.63171,1.63409,108665
22/12/2013,22:00,1.63346,1.63741,1.63200,1.63515,1.63442,1.63762,1.63257,1.63615,82600
23/12/2013,22:00,1.63515,1.63806,1.63211,1.63671,1.63615,1.63825,1.63242,1.63709,61296
chimpy
FXCodeBase: Initiate
 
Posts: 112
Joined: Fri Jun 10, 2011 10:32 am

Re: Live chart data export to CSV

Postby Gennadiy » Sun Jan 19, 2014 11:32 pm

Hello chimpy,

Thank you for report. We will check the issue on nearest Thursday and Friday.
I will let you know about results.

Thank you.
Gennadiy
 

Re: Live chart data export to CSV

Postby chimpy » Mon Jan 20, 2014 8:38 pm

Gennadiy,

I dont know if its really going to help you observing it real time on thursday and friday, considering all the historical data is the same going back decades... The application did not produce results this before the API caused the app to break, as I remember using it for live trading. I am not sure if the problem is related to the tradingdayoffset, tradingweekoffset flags, i never really understood why they are in the config file or how they work?

I am sure the data from teh fxcm server is correct, its how the app is pulling it down and dating it up thats the problem
chimpy
FXCodeBase: Initiate
 
Posts: 112
Joined: Fri Jun 10, 2011 10:32 am

Re: Live chart data export to CSV

Postby Gennadiy » Thu Jan 23, 2014 1:47 am

Hello chimpy,

We have analyzed the data and understood, that all works well:
12/12/2013,22:00

The date is the time of beginning of the bar/candle. So the sentence above means that the bar is opened in 22:00 on the Thursday. But the bar will be closed on Friday. So the bar includes the data of Thursday and Friday.

We will check it on Friday. I will let you know about results.

tradingdayoffset, tradingweekoffset flags

Unfortunately allpin did not leave any documentation according these parameters. I have tried to understand what do they mean from the sourcecode, but without success. Our developers do not know too.
Gennadiy
 

Re: Live chart data export to CSV

Postby chimpy » Thu Jan 23, 2014 12:40 pm

Gennadiy,
Allpin used to work for the company so I think someone can get hold of him by email or professional website like linked-in I am sure he will remember because we communicated a lot by email but it was the company address.

I have not adjusted either of those offsets since the last time he told me to chance them in the development process as listed in the thread.

The bars are not correct. Because firstly they are not consistant with Marketscope. See the attached picture. The Bar on the 8 November 2013 exists but does not exist in the app data.
For example as of the time of writing its 23rd Jan and that bar has been on my marketscope since 22:00 on 22nd Jan. It will represent 23rd of Jan when its closed, even though it started on 22nd of Jan at 22:00.

I was using this app as my main data source before it died when the API changed and there was always a friday bar, exactly the same as marketscope.
Attachments
f.jpg
bar
chimpy
FXCodeBase: Initiate
 
Posts: 112
Joined: Fri Jun 10, 2011 10:32 am

Re: Live chart data export to CSV

Postby Gennadiy » Fri Jan 24, 2014 4:39 am

Hello,
I have wrote an email to allpin with the question about config parameters. I will let you know about the answer.

Please take a look at attached image.
Compare the data from the Marketscope and CSV file created by allpin's program.
Marketscope shows the candle on 01/17/2014, but the program shows the same candle on 01/16/2014 5:00:00 PM. So all works well, because of Marketscope draws candle on the day when it's closed. But the program writes the date and time when the candle opened.
Attachments
D1.png
Gennadiy
 

Re: Live chart data export to CSV

Postby Gennadiy » Thu Mar 27, 2014 4:59 am

Hello chimpy,

Please find the new version of the application below.
LiveChartDataExportToCSV_v2_x86.zip

LiveChartDataExportToCSV_v2_x64.zip


Release Notes:
1) new configuration file;
2) new format of date in CSV (by analogy with the FX Trading Station);
3) The issue with duplication of bars was fixed;
4) CSV updates with each new tick (not a timer as in the previous version).

Thank you.
Gennadiy
 

Re: Live chart data export to CSV

Postby chimpy » Thu Jul 24, 2014 12:46 am

Gennadiy,


I havent been able to look at this for a few months, apologies.

>>2) new format of date in CSV (by analogy with the FX Trading Station);

Below is some data form GBPUSD. You can see that on the 17th Thursday the market is closing for the weekend whereas marketscopes same candle shows its the 18th a Friday. This is the same problem we had before.
(marketscope and the script are both set flags as UTC)
Code: Select all
15/07/2014,21:00,1.71404,1.71498,1.71114,1.71338,1.71464,1.71524,1.71151,1.71411,119200
16/07/2014,21:00,1.71338,1.71426,1.70843,1.70987,1.71411,1.71450,1.70866,1.71040,205346
17/07/2014,21:00,1.70987,1.71167,1.70352,1.70789,1.71040,1.71190,1.70378,1.70909,131131
20/07/2014,21:00,1.70805,1.70984,1.70547,1.70736,1.70950,1.71009,1.70571,1.70777,91928
21/07/2014,21:00,1.70736,1.70822,1.70404,1.70626,1.70777,1.70847,1.70426,1.70667,93473
22/07/2014,21:00,1.70626,1.70942,1.70229,1.70415,1.70667,1.70967,1.70257,1.70459,105142
23/07/2014,21:00,1.70415,1.70416,1.70303,1.70308,1.70459,1.70465,1.70328,1.70335,10292


>>4) CSV updates with each new tick (not a timer as in the previous version).

I see all the time frames under 1 day are updating in real time. The D1 W1 and M1 are not updating in realtime. They only update when you restart the script. I vaguely remember this problem happening before on this timeframe.

I'd you could have a look I'd appreciate it. Thanks again
chimpy
FXCodeBase: Initiate
 
Posts: 112
Joined: Fri Jun 10, 2011 10:32 am

Re: Live chart data export to CSV

Postby chimpy » Thu Jul 24, 2014 1:02 am

To add: You mentioned the application updates on a new tick and not the timer anymore so I removed the <Timeout>1</Timeout> tag and run the application again, I then got this messsage

Exception System.NullReferenceException: Object reference not set to an instance of
n object.
at fxcore2.Samples.Program.Main(String[] args) occured
Press any key to exit

so it still wants the <Timeout>1</Timeout> tag?
chimpy
FXCodeBase: Initiate
 
Posts: 112
Joined: Fri Jun 10, 2011 10:32 am

PreviousNext

Return to Custom Indicators

Who is online

Users browsing this forum: No registered users and 73 guests