You system tend to reward fluke success while not taking into consideration overall performance. In your system a guy who has made 4 maps rated 4.3 is worse than a guy who has made 10 maps rated below 2.0 and one 4.4. This doesn't mean that it is bad, it just means you compare one persons best performance against other persons best performance, while ignoring low performance."

Actually, in my system, the guy with 4 maps rated 4.3 has a much higher score than the guy who made one map rated 4.4 (under the assumption all maps have a similar number of votes).

* 25*(4.3-3.5)*V/(V+100) + 18*(4.3-3.5)*V/(V+100) + 14*(4.3-3.5)*V/(V+100) + 10*(4.3-3.5)*V/(V+100) = 53.6 * V/(V+100)
* 25*(4.4-3.5)*V/(V+100) = 22.5 * V/(V+100)

Clearly, the first person has a score that almost doubles the score of the second person. The second person would need a lot more votes on his map to get a higher score: if the first guy has more than 70 votes on his maps, the second guys can't get a better score no matter how many votes he has.

p.s. It just occurred to me while writing this, that this discussion we are having has no point. I don't care about your lists ( or anyone's for that matter ) and in the end it's your rating system do with it what you will :D.

I do value all feedback though (positive and negative). It can only help to improve the metric in the future.

Edited 11/15/2014 13:41:17

To check the maps that someone made, go to the Map page (under home), select all maps and then Ctrl+F for the name. For Nathan:

http://warlight.net/Map?ID=13539

http://warlight.net/Map?ID=13317

http://warlight.net/Map?ID=13152

http://warlight.net/Map?ID=13647

http://warlight.net/Map?ID=13774

http://warlight.net/Map?ID=13777

Great! We're good then :D. I guess I was wrong, my apologies :). But I still think you should include all maps.

Nathan made 6 maps! I didn't know that.

Looks like i just about made the top 15% of mapmakers.

Edited 11/15/2014 17:28:01

Also does anyone else notice that the longer a map stays on warlight the further ratings go down.My top map had dropped from the first page to 3rd over time.I have seen this in all maps.How does this equation deal with the overtime quality downfall for maps.

Edited 11/15/2014 17:30:27

It could be that the quality of new maps are better as time goes on or it could be largely due to popularity. Another idea is that as new players join the overall taste of the community shifts as well.

@Ironheart

Here's my theory:

When a map is just published it appears on the dashboard, and the first players who rate it can be divided into two groups. One is the mapmaker himself and his friends, who will give it very good ratings. There's also the map enthusiasts, who will want to get the map to 10 ratings and only glance over it or play a single game. Their ratings will be more extreme, either really high or really low, as they have not given it much consideration.

But after the ten ratings mark is reached, the map will probably enter the front or second page, after which it will be discovered by other players who are curious for its high rating. They will start playing it more thoroughly, and give more accurate ratings.

2 of my maps were rated as the best map for a couple hours, and now my best is at the end of page two. But that's just how things go, maybe one day I will make something that stays on the front page for a little while longer. This ranking list sure gives me the incentive to do so, maybe I can get myself into the top 20 :)

Edited 11/15/2014 17:59:32

Hmmmm, wonder if I'm actually on the list or if my maps are too low rated to even make it there :P

Honestly, it seems that maps peak in rating when they're new, and then stagnate to their proper position after a few months. Seems to have happened with my maps. :/

I subscribe to Moros' theory, makes a lot of sense.

The "beauty" of my method is that the vote-corrected score will be similar when this happens and in most cases still slightly increase when more votes are cast as long as the average vote remains clearly above 3.5.

As an example, consider the following fictional evolution from map with a "real" average around 3.7 while the first 10 voters gave it a higher vote:
* 10 votes - 4.5 rating: B=1.0, S=1.0*10/(10+100)=0.0909
* 20 votes - 4.1 rating: B=0.6, S=0.6*20/(20+100)=0.1000
* 40 votes - 3.9 rating: B=0.4, S=0.4*40/(40+100)=0.1143
* 80 votes - 3.8 rating: B=0.3, S=0.3*80/(80+100)=0.1333
* 150 votes - 3.75 rating: B=0.25, S=0.25*150/(150+100)=0.1500
* 400 votes - 3.7 rating: B=0.2, S=0.2*400/(400+100) = 0.1600



@ Riyamitie: your score and rank were added: 59th. Your highest rated map alone (Aselia) would still have you ranked 70th, you had 3 qualified maps ("2009 US Core Based Statistical Areas" and "Water Earth" being the others).

Yes Riyamitie you are rated 59th!!! just be happy you are not 70th. Sit in your corner and let Math Wolf decide.

i wonder where i am

In hell? Hopefully!(I hated the Kate Upton map)

hahahaha be nice qwed117

Still better than Qwedlandia

Edited 11/16/2014 08:59:57

aaaaand definitely better than that diarrhea map

Edited 11/16/2014 08:59:51

y u do dis

Wow, what a nice job, I don't really like to "rank" mapmakers and that kind of things, but I have to recognise that this is an amazing job, and really accurated.
It would be nice to know if I am round there, my maps are not good, but I think I have some with more than 3.5 .

That's nice summary :) But I would change something:

I also think, that you should take into account all maps made by each mapmaker.
About "Using all maps penalises map makers who have more, but some not very high quality maps.", Isn't that's why you added this "T" factor? With walue of 3 for map nr 10, overall rating is not affected by bad maps too much.

Also I'm not sure if making "T" fixed is ok. It penalises map makers, who focus on quality instead of quantity. So someone, who made 5 good maps (let's say rated 4,3), can have lower score, than someone, who made 2-3 good maps, and 7-8 average.

like:
1st person: 4,3*25+4,3*18+4,3*14+4,3*10+4,3*8=32,2
2nd person: 4,5*25+4,4*18+3,8*14+3,8*10+3,7*8+3,7*7+3,65*6+3,6*5+3,55*4+3,5*3=40,3

(Let's omit "S" factor for this purpose assuming all maps are rated 1000 times or so)

So my suggestion is to use following weighted factor:



where n - no of maps someone made, i - ordinal number

Explanation for ppl who are not in Statisticians clan:
if someone made 1 map T=1
with 2 maps, T1=2/3 T2=1/3 (1+2=3)
3 maps: T1=3/6 T2=2/6 T3=1/6 (1+2+3=6)
(...)
10 maps: T1=10/55 T2=9/55 T3=8/55 (...) T10=1/55 (1+2+3+4+5+6+7+8+9+10=55)

The problem is with "effective gathering data" I guess, so you can limit it to for example 10 maps per person.
Also yes, it means, that someone, who made 1 map with highest rating will be 1st on your list, but hey, try to make only 1 map, that will be masterpiece :)

Edited 11/17/2014 18:54:38