Excel automatically drops precision from .csvs

[akmiller01]

I've just noticed an issue that could be problematic (and hopefully hasn't come up yet). When editing a .csv, for numbers with extremely large values, Excel decides to truncate the value without warning.

E.g.: 72e1e39

In this case, I just changed the title names and saved. When I opened up GitHub, I noticed that two rows had also been edited. It automatically changed 1.30835x10^11 to 1.31x10^11. In this case, a difference of $165,000,000. The only way I was able to edit the column headings without having excel automatically drop the precision was by using Notepad. I don't know how we can ensure this hasn't happened in the past, and I'm not sure what the best way is to prevent it from happening in the future.

[akmiller01]

@bill-anderson @timstrawson This has been happening since May. Look at the change-log for country-year/gdp-usd-2005.csv in this commit: 85adaa0.

Brazil 2005 starts on line 122 and ends on line 106. In 122 it's equal to 882,186 million, or 882,186,000,000... On line 106 (and in the present file) it's equal to 8.82*10^11 or 882,000,000,000 a difference of 186 million.

[akmiller01]

As a matter of principle, I think I'm going to stick to using LibreOffice (http://www.libreoffice.org/) Not only does it not arbitrarily decide to truncate large values in .csvs, but it also seems to fix some of the terrible things Excel has done to our data: eba5351.

That being said, it won't be able to save the precision that has already been dropped.

[akmiller01]

Well, just did some quick checking, and the good news is only GDP and long-term-debt have values large enough to trigger this kind of truncation. That being said, we might want to revisit our calculations for GDP.

[dw8547]

@dw8547 to check GDP files although not coming in through Excel.