Apologies for the late reply, I was mostly playing at work where I didn't have the file with the results with me.
@ Bane: not ranked, you do not have a map rated higher than 3.5 when I gathered the results.
@ alex23: I got you at 121 with 4 maps rated over 3.5.
@ Latnox: I also think, that you should take into account all maps made by each mapmaker.
About "Using all maps penalises map makers who have more, but some not very high quality maps.", Isn't that's why you added this "T" factor? With walue of 3 for map nr 10, overall rating is not affected by bad maps too much.
Whether or not to include all maps is of course a subjective argument. It is certainly true that it would not be affected much by bad maps. My counterargument would be: is it right that someone who made 1 map, rated below 3.5 has a negative score, lower than someone who never even tried to make a map? Still, I may adopt this in updates.
Also I'm not sure if making "T" fixed is ok. It penalises map makers, who focus on quality instead of quantity. So someone, who made 5 good maps (let's say rated 4,3), can have lower score, than someone, who made 2-3 good maps, and 7-8 average.
Again a subjective argument, but I can follow the reasoning. In your example, I would indeed prefer that the second person scores similarly, possibly better than the first person. The reason is that I want to measure success and if those 7-8 average maps still score sufficiently above 3.5 and are played often, I think they should be valued for it.
In your specific example however, the first person would get a higher score than the second (60 vs. 53 with equal, very large number of votes). From what I understand, you don't take into account the -3.5 in your calculation. This makes a huge difference as it further downweights the influence of the lower rated maps.
I do really like your idea for a weighting factor based on the number of maps. I'm not completely agreeing with its construction, but I could implement a similar concept. I'd rather give weights of a shape 1/(i+2) for example (divided by the sum over all 1/(i+2) of course). The reason is that I do not want a linear decrease in weights.
Additionally, I do want some correction for a low amount of maps. We can already see that people with 1 map can rank relatively high. Not correcting for this may lead to having someone like Vampires - Kayn rank first (or at least top 3), which wouldn't make much sense either. I'd solve this by adding an "empty" map maybe. So divide by the sum of 1/(i+2) with i going from 1 to n+1 for example (with n the number of maps). Or, to get a similar result as the one I have now, it would be 1 to n+4.
The problem is with "effective gathering data" I guess, so you can limit it to for example 10 maps per person.
This is part of the problem, but there are solutions for that if I cut it off to 100 mapmakers for example as well.
Thanks a lot for that feedback, it is greatly appreciated! Can I contact you when I plan to repeat this exercise in a few months?
@ Green;I've considered undertaking similar projects with Warlight data myself, but have always been put off with the data collection.
Did you collect this massive amount of data manually, or automate it? If you automated the process what method did you use?
I know how you feel. I have many other projects in mind, but the data collection is always the weak point. This one was done manually, took me about 7 hours to gather all the data.
Edited 11/22/2014 15:57:09