Pages

Monday, February 3, 2020

Garbage In, Garbage Out

When I woke up Saturday to find a message on Discord from Rhivre, the editor-in-chief of INN, asking to discuss a recent blog post, I knew I'd fucked up. Especially when the blog post involved looking at data from the Monthly Economic Report. Basically, the data on my graphs labeled as coming from June 2019 did not come from June 2019. Crap.

Rhivre thought I'd merely mislabeled the data. But a quick investigation discovered something worse: I'd pasted the data from June 2018 into my data set as June 2019. So instead of comparing activity right before the null sec communications blackout began with the end of the year, I was comparing data from 18 months previously.

The mistake basically was a misclick. I opened the wrong file to copy and paste out of. The RegionalStats.csv file included with every MER only contains that month's regional data. In order to have one data set, I copy the contents of the file into a spreadsheet every month and use that to run my scripts against. Fortunately, June 2019 was the only month affected.


But really, I made another mistake. I shouldn't have compared a summer month to a winter month. I should have compared December 2019 to December 2018. The graph still shows Delve ridiculously far out in front in December 2018. Also, the tremendous shrinkage in the value of the PvE activity is still clear. But, I would have accounted for seasonal variations comparing one December with another.


Even with the mistake, I don't intend to give up performing analysis of the New Eden economy. Like a lot of EVE players, I like graphs and charts. I also need to practice R skills like creating charts and graphs. In my classes, I'll go a month at a time without creating a data visualization and then at the end of the class, I have to go through my notes to see how to create a graph.

In the future, I'll try to produce graphs at a more cluster-wide level. I do have an idea for producing graphs at the Empire level for all the high and low sec players. I just need to make sure I have everything correct, including the data. As the old saying goes, garbage in, garbage out.

No comments:

Post a Comment