Live chart data export to CSV

Here you can post and download custom indicators. PLEASE: Do not start topics unless you are posting your own indicator, they will be moved to appropriate section even if you do.

Moderator: admin

Re: Live chart data export to CSV

Postby chimpy » Wed Aug 03, 2011 4:44 am

Hi Allpin,
Another Issue I left out above,

The W1 D1, H6, H4 and H3 timeframes are not updating after the initial history download.

The m1, m5, m15, m30 and H1 timeframes seem to be downloading correctly. the H1 for example will see close and volume updates every 60 seconds (as per the timer)

Thanks
chimpy
FXCodeBase: Initiate
 
Posts: 112
Joined: Fri Jun 10, 2011 10:32 am

Re: Live chart data export to CSV

Postby allpin » Wed Aug 03, 2011 7:33 am

Hello Chimpy,
I suspect that the issue with M1 timeframe is not a problem. If the configuration file asks, for example, for 1000 monthly candles, it is likely that history was not gathered for that long (almost 20 years). Try to decrease the "NumBars" value.
Hope it helps!
allpin
FXCodeBase: Confirmed User
 
Posts: 21
Joined: Thu Aug 19, 2010 3:29 pm

Re: Live chart data export to CSV

Postby allpin » Wed Aug 03, 2011 7:56 am

Hello Chimpy,
Could you please verify the issue with “big” timeframes like “W1”, “D1”, “H6”, “H4”, and “H3”? The last line (last candle) should be updated on timeout. Let’s look on the most likely situation when the last candle is not closed yet (for example, we have timeout every 60 seconds and we are looking for “D1” (daily) bars). After timeout the open price would be the same, the high and low price could be the same, but the close price most likely to be changed, and the volume should be increased.
allpin
FXCodeBase: Confirmed User
 
Posts: 21
Joined: Thu Aug 19, 2010 3:29 pm

Re: Live chart data export to CSV

Postby chimpy » Wed Aug 03, 2011 4:18 pm

allpin,
Thanks, The M1 timeframe had the same filename as the 1 minute, thats what caused the clash.
Its working fine now thanks :)


---
I am using the the following values for the timeoffset
<TradingDayOffset>-7</TradingDayOffset>
<TradingWeekOffset>-1</TradingWeekOffset>

These give me EST time. I have tried many different values but I always get EST time values in the CSV files.

----

Thats correct, the most likely thing to happen wioth the larger timeframes is that the close and volume will change every timeout value. over an hour I did a study yesterday of all the available timeframes. Anything under an hour works properly. The H1 works properly, anything larger than H1 does not update the volume or close.
I noted the last line by reading the CSV file manually, copied the values down then went back to them some minutes later to see if it had changed, thus ruling out any charting package problem.

Thanks
Chimpy
Last edited by chimpy on Thu Aug 04, 2011 7:55 am, edited 1 time in total.
chimpy
FXCodeBase: Initiate
 
Posts: 112
Joined: Fri Jun 10, 2011 10:32 am

Re: Live chart data export to CSV

Postby chimpy » Wed Aug 03, 2011 7:53 pm

Allpin

correction, H3 is working ok.

Something else I noticed, is that at the faster timeout values like 1,3,5 seconds there is obviously a lot of disk hashing with the app accessing the temporary files that form in the c:\windows\temp directory and updating them to the CSV files. at these fast refresh rates the app naturally has to do a lot of work accessing hardware and this causes the initial history download to show down as well as the disk hashing.
so I was thinking, can a variable for the temporary file directory be added so that I can point this to a RAM drive instead of the windows\temp default, this will allow the app to access the temporary files much faster.

Otherwise I was amazed how lean and mean this little application is.. It uses less than 1% of CPU time even when you set it to update every 1 second.
chimpy
FXCodeBase: Initiate
 
Posts: 112
Joined: Fri Jun 10, 2011 10:32 am

Re: Live chart data export to CSV

Postby allpin » Thu Aug 04, 2011 8:15 am

I have another suggestion about problem with writing file with
timeframe "M1" (one month). Is it possible that you have program
writing file for time frame "m1" (one minute) at the same time?
So if you have something like this in your configuration file:
Code: Select all
    <History>
      <Instrument>USD/JPY</Instrument>
      <TimeFrame>m1</TimeFrame>
      <File>USD_JPY_m1</File>
      <NumBars>1000</NumBars>
    </History>
    <History>
      <Instrument>USD/JPY</Instrument>
      <TimeFrame>M1</TimeFrame>
      <File>USD_JPY_M1</File>
      <NumBars>10</NumBars>
    </History>

That could be a problem because for Windows "USD_JPY_m1.csv" and
"USD_JPY_M1.csv" is the same file.
Anyway I can see possible solutions here:
1 - change the "File" value in the configuration file for "M1", for
example:
<File>USD_JPY_OneMonth</File>
2 - change the OutputDir to another value (create a new directory,
or point to the directory where file "USD_JPY_M1.csv" does not exist
yet).
3 - try to delete existing file "USD_JPY_M1.csv" (you would not be
able to do it if the file is really in use)
allpin
FXCodeBase: Confirmed User
 
Posts: 21
Joined: Thu Aug 19, 2010 3:29 pm

Re: Live chart data export to CSV

Postby chimpy » Thu Aug 04, 2011 11:18 am

Thanks allpin, I did notice that with the M1 this morning and fixed it myself, yeah the m1 and M1 caught me out..


Something I have noticed on the H4 history. From about May 2011 going back to 2002, Open and Closes are back to front. This has the effect of giving Bull candles a red body and Bear candles a Blue or Breen body. I have attached a sample of the data plus a picture showing the problem. After May 2011 the problem goes away and the candles look normal.

This problem is probably nothing to do with the application but, someone handling the data should look into it so, I thought I should mention the problem. strangly though, the H4 candles in marketscope look correct?
Attachments
strange.candle.jpg
picture
4hgbpusd.sample.csv
csv sample
(6.67 KiB) Downloaded 826 times
chimpy
FXCodeBase: Initiate
 
Posts: 112
Joined: Fri Jun 10, 2011 10:32 am

Re: Live chart data export to CSV

Postby chimpy » Thu Aug 04, 2011 11:32 am

In reference to the back to front Open and Closes above, I have just had a look at the USDCHF H4 and the same thing is happening before May 2011 approximately.
chimpy
FXCodeBase: Initiate
 
Posts: 112
Joined: Fri Jun 10, 2011 10:32 am

Re: Live chart data export to CSV

Postby allpin » Fri Aug 05, 2011 9:07 am

Hello Chimpy,
Please have a look at the issue of .csv file not updating after
loading initial prices. I would like to ask you to use this value:
<TradingWeekOffset>0</TradingWeekOffset>
Please let me know whether it would fix the problem.
Your approach to check the updates is right: open the files (as read
only), write down values, close the files, wait (at least for timout),
and then repeat the sequence several times.
allpin
FXCodeBase: Confirmed User
 
Posts: 21
Joined: Thu Aug 19, 2010 3:29 pm

Re: Live chart data export to CSV

Postby chimpy » Fri Aug 05, 2011 1:37 pm

Allpin,

I changed the trading week offset to 0. Noting happened, same problem.

I thought the offsets have something to do with the Timezone anyway?
In any even they do not change the timezones either. Still stays as eastern standard time, whatever you set them to.. Is this something that needs fixing or are you able to change the time zones yourself with your test app?


Did you see my pic with the upside down candles? whats causing that? does fxcm know their data is like this?
chimpy
FXCodeBase: Initiate
 
Posts: 112
Joined: Fri Jun 10, 2011 10:32 am

PreviousNext

Return to Custom Indicators

Who is online

Users browsing this forum: Google [Bot] and 33 guests