<< Back to Map Development Forum   Search

Posts 1 - 30 of 79   1  2  3  Next >>   
BEST (most successful) mapmakers: 11/13/2014 22:54:54


Math Wolf 
Level 63
Report
Since most lists about maps are based on quantity, I have developed a metric that summarizes quality and success for map makers, which includes quantity as well.
The exact formula (beware math) is below.
The final results are as follows:

1	RA			43.558
2	Incaman			42.998
3	Срећко			40.829
4	Fizzer			38.125
5	[中国阳朔]Chaos		32.633
6	Major Risk		30.994
7	Issander		30.741
8	The King [Belgian G...	30.638
9	Roblexa Caritelisa 948	28.971
10	{rp} pedrito		28.401
11	Ottaman			26.580
12	Troll			25.847
13	Caia Veloso		25.441
14	Qi			22.770
15	Saruman			22.537
16	Grundie			22.021
17	Apollo			21.693
18	Kheimon			20.261
19	Muppet			19.953
20	Vampires – Kayn		19.824
21	Moros			18.292
22	Dameon			17.976
23	[中国阳朔]TexasJohn	17.500
24	King Philip		17.339
25	Nate			16.301
26	C.K – 胖鹌鹑		16.216
27	eliod			15.999
28	DynamiteT		15.622
29	Dogrosebush		15.140
30	Phil			14.722
31	Smok			14.275
32	Koger			14.211
33	Arc Light		14.179
34	Ranek			13.831
35	NinjaNic		13.702
36	Huruey • apex		13.622
37	Urfang			13.621
38	sue			13.537
39	muligan			12.864
40	Megli			12.498
41	Guderian		11.611
42	Smorgborg		11.365
43	Gunz			11.315
44	Worldbeing		10.214
45	Traintown		10.079
46	Opum's Razor		9.817
47	Lord Syvedyas		9.705
48	Yeezus Theory		9.680
49	kendou			9.283
50	Jon Snow		9.233

59	Riyamitie		7.452
61	[NL] Willem van Oranje	7.297
66	Metaltubbie		6.561
67	Frankdeslimste • apex	6.553
69	skunk940		6.311
70	Clement Attlee		6.280
72	Nathan			5.882
137	Kroma			1.966
148	ChrisCMU		1.628
163	Bananashake		1.125
168	Sir Nick		0.992

*229 mapmakers were ranked. Extrapolating, I'm guessing there are at least ~400 mapmakers in total.

For the formula:
  • Only maps with a rating higher than 3.5 were considered.

  • Define the base score "B" of each map as its rating substracted by 3.5. This generates scores between 0 and 1. (Only maps with ratings higher than 3.5 were considers, no maps have ratings higher than 4.5).
    Why 3.5? Because I only want to include maps with decent quality. Maps with ratings over 3.5 must have gotten at least half of the time a vote of 4 or regular votes of 5 stars. I found 4 too high as there are still many decent, often played, maps with ratings between 3.5 and 4. Also, at a first glance, the average and median ratings of maps both seem to be close to 3.5.

  • Use V as notation for the number of votes for a map. The vote-corrected score S = B * V/(V+100). Thus, the score S is always higher than 0, and always lower than B. When a map has a large number of votes, S is close to B. When this number is low, S is closer to 0.
    Why do I do this? There are two reasons: (1) more votes means a more reliable score. I want to avoid inflation from maps who just received some 10-15 votes of all the mapmaker's friends or alts. In general, the rating of (highly ranked) maps often tends to decrease when more votes are cast. Technically, I am adding 100 votes of "3.5", fictional users who pull the rating towards the generally expected mean. This could also be interpreted as a prior distribution. (2) more votes means more plays and more success. I don't just want to measure quality. You can have the best map on WL, but if nobody plays it, the success is limited. By downweighting with 100 fictional votes, I guarantee that maps who are rated often get most of their base score B in their corrected score S, while maps that are hardly played, only received a fraction of the base score.
    Example: 11 votes: S=B/10, 25 votes: S=B/5. 50 votes: S=B/3, 100 votes: S=B/2, 200 votes: S=2B/3, 900 votes: S=90% of B, over 2000 votes (most played maps): S > 95% of B (96.4% for the most played map).

  • The score of each map maker is the weighted sum of his best 10 maps with ratings over 3.5. (Only 7 mapmakers actually had 10 or more maps with a rating over 3.5, for all others, only the available maps were counted). The weights were as follows: 25, 18, 14, 10, 8, 7, 6, 5, 4, 3. So the total score T=25*S1+18*S2+14*S3+... These weights do sum to 100, so technically, the maximum score that could be reached would be 150 (if a mapmaker had 10 maps, all ranked 5 with a huge amount of votes). In practice, I expected no mapmaker to score more than 50.
    Why those weights? I thought it was important to give all mapmakers a fair chance, even those with a relatively small amount of maps. Thus, a few maps should carry most of the weight. Specifically, I wanted the first map to count as a quarter of the total score as this is the "masterwork". The first 3 maps cover more than half, with the first four maps covering over 2/3rd of the total score and the first half exactly three quarter (which implies that maps 2-5 count for half, 6-10 for a quarter and the first for a quarter).

  • Note that the scores of the maps are sorted by vote-corrected score S and not by base rating B! As an example: Opum's Razor has a map with a rating of 3,9716 and 141 votes, so B=0.4716, V=141, S=0.276. Another map has a rating of 4.0000 and 48 votes, so B=0.5, V=48, S=0.162. Thus, the first map is given weight 25, the second (although with a higher rating), is given weight 18. The other way around, he would have been penalised for having his lower ranked map played more and his total score T would have been 9.021, which would have dropped him to 51st place.

  • Summary of the formula: http://www.texpaste.com/n/zvr6xxne

  • Scores were collected on November 11-November 13 and thus may show small innaccuracies. All scores above 4 were collected on the same day.


If anyone who was not ranked wants to know their rating and/or rank, feel free to ask.

Edited 11/15/2014 22:07:06
BEST (most successful) mapmakers: 11/13/2014 22:55:36


Math Wolf 
Level 63
Report
Discussion of the results:

  • RA wins it, but just barely. The difference between RA and Incaman is obvious. While Incaman has 3 of the highest rates maps on Warlight, including the single highest rated map (Rise of Rome: 4,4448, S=0.901), RA beats him by the quantity of his quality: 7 maps rated 4 or higher with one extra rated 3.8896. No other mapmaker has more than 4 maps rated higher than 4. (although Apollo and Roblexa Caritelisa 948 both have 4 maps with rating higher than 4 and 1 with a rating of exactly 4 at the moment the data were gathered.)
  • Срећко combines top quality (Europe Big, Modern Europe both rated over 4.3) with quantity (9 maps rated over 3.5). After his 3 highest rated maps, most aren't played very often, so he still has potential.
  • Fizzer basically surfs on the success of all the Earth adaptations. Most of those are both highly and often rated. Medium Earth leads the way, but is overall only the 5th most rated map! (behind Earth, Europe, Rise of Rome and Imperium Romanum)
  • [中国阳朔]Chaos is maybe the most consistent quality mapmaker around. I lost count of the number of maps in the 3.7-4.1 region with ~100 ratings that he has.
  • Major Risk gets most of the success of the European Borders map, followed by 3 other highly ranked, but somewhat less played maps.
  • Issander's highest rated map is hardly played, but his Huge World makes up for that. Many maps with high ratings, but some have very few ratings.
  • The King, similarly to Issander combines the result of 2 highly rated maps (almost equal S score) with many other good quality maps.
  • Roblexa Caritelisa 948 has a large amount of good quality maps, some of which are not often rated (including the highest rated map).
  • Both {rp} pedrito and Ottaman have 2 highly rated, often played maps.
  • Based on only the Europe map, Troll would still be ranked 15th.
  • Caia Veloso, Kheimon, Vampires - Kayn, Dameon, [中国阳朔]TexasJohn and some others get most, if not all of their success from one highly rated and often played map.
  • Qi, Saruman, Apollo, Muppet and Moros use decent quantity on the other hand. Apollo has over 10 qualified maps, but none of them is rated more than 100 times.
  • Grundie equals Earth and small Earth. His maps are rated a combined 4400 times!


Edited 11/14/2014 10:03:24
BEST (most successful) mapmakers: 11/13/2014 23:00:41


Red Menace
Level 55
Report
I have a few projects in the making that will hopefully bump my very low score up.
BEST (most successful) mapmakers: 11/13/2014 23:07:16


Жұқтыру
Level 55
Report
Go, Statistician clan!

This is all looks like complex work, thanks! Now we finally know who is actually best.

Слава Срећки!
Слава Ра-у!

And, also we know how self-inflated some people really are (skunk940...).

Edited 11/13/2014 23:07:58
BEST (most successful) mapmakers: 11/13/2014 23:51:03


Phulesdorp
Level 28
Report
{Wolf whistles}

That's a big list! I'm probably going to have to take a look at RA's maps after this.

... But I would really enjoy the formula more if it was presented just a little more elegantly (a very good example for this is Euler's identity). But if you don't, it's fine!

... It's just that I'm a sucker for nice-looking formulas. :p

Edited 11/14/2014 00:00:15
BEST (most successful) mapmakers: 11/14/2014 00:00:26


ScarlettTD 
Level 57
Report
Nice work!

Thanks Math Wolf, a thorough analysis and a great read.
BEST (most successful) mapmakers: 11/14/2014 00:16:44


Math Wolf 
Level 63
Report
@ Phulesdorp: http://www.texpaste.com/n/zvr6xxne
@ Yeezus Theory: I personally feel every point above 0 is a contribution to WL as it means you made at least one map rated higher than 3.5. So I would never call your score low. I think it's not that easy to get into the top 50 of this ranking.
BEST (most successful) mapmakers: 11/14/2014 00:30:43


Mudderducker 
Level 57
Report
This moment in time my maps are overrated, the biggest amount of ratings is only 91. I've yet to prove myself. I still have a couple of maps to try and make widely playable.

My point, the formula is good. But doesn't take the amount played into consideration. Though I can't say I keep track of how many games are played on my maps. But I can say only about 10-20 maps are played frequently.

Edited 11/14/2014 00:37:30
BEST (most successful) mapmakers: 11/14/2014 00:39:10


Phulesdorp
Level 28
Report
Nice. :)
BEST (most successful) mapmakers: 11/14/2014 00:40:14


Beren • apex 
Level 63
Report
@Apollo, it looks like he's using number of ratings as a proxy for number of plays, since we don't have that data at this point. Hopefully Fizzer will release that information, at which point we could incorporate that into a new formula for this.
BEST (most successful) mapmakers: 11/14/2014 03:44:26


Nate
Level 18
Report
Twenty fifth. hmm guess I better keep working.
BEST (most successful) mapmakers: 11/14/2014 07:37:56


skunk940 
Level 58
Report
Good maths
BEST (most successful) mapmakers: 11/14/2014 10:36:15


Math Wolf 
Level 63
Report
@ Beren Erchamion (& Apollo): That's mostly correct indeed. I do need the number of votes itself as those define the accuracy and precision of the rating (reason 1). But for the success (reason 2), it would be better to have the actual number of plays. Ideally, that part of the formula would be a product of terms including the number of votes and number of plays rather than just the votes.

As far as I know, there is no data on the number of plays of a map and guesstimates are very difficult to obtain, I think. So I feel that for average to highly rated maps, the number of votes is as good a proxy as any at this moment. (For lower rated maps, I think it would be a bad estimate as people would play once, vote and never play again.)

Additionally, multiplayer games get saved and stored and thus can be counted (technically), but single-player games not. (They do not have an ID!) So it may actually be impossible to retrospectively count the total number of plays of a map. It should be relatively easily (server side, crawler) to count the number of multiplayer games of a map.
BEST (most successful) mapmakers: 11/14/2014 13:40:24

An abandoned account
Level 56
Report
Well, Skunk is only 0.031 points ahead of me despite the fact he's made more maps than anyone and I've only ever made 5.
BEST (most successful) mapmakers: 11/14/2014 14:24:57


Moros 
Level 50
Report
Wow, it's incredibly cool to see myself at place 21 of such a ranking, let alone one made using solid maths. Thank you!
BEST (most successful) mapmakers: 11/14/2014 18:44:20


ChrisCMU 
Level 60
Report
I'd like to know where I rank. I think I only have 4 maps done, but all are well made IMO.
BEST (most successful) mapmakers: 11/14/2014 20:02:13


Phulesdorp
Level 28
Report
So... from what I'm understanding, it's practically a Simpson's paradox to all "unpopular" maps? That's slightly discouraging for me and my mapmaking skills. :(

And will this list ever get expanded upon in the future (since I don't know what a crawler is, I'm taking it as a good sign)?
BEST (most successful) mapmakers: 11/14/2014 20:54:28


Incaman
Level 56
Report
Interesting :D but your whole value system is a bit flawed.

Edited 11/14/2014 20:55:07
BEST (most successful) mapmakers: 11/14/2014 21:55:16


Ranek
Level 55
Report
Thats right. next time, take care that incaman is the number one!^^
BEST (most successful) mapmakers: 11/14/2014 22:09:30


ChrisCMU 
Level 60
Report
I find this fascinating. Out of the top 50, I probably didn't even know 1/2 of those people made a map at all. It means either the rating system is flawed or they have great maps I need to check out.
BEST (most successful) mapmakers: 11/14/2014 22:27:45


Incaman
Level 56
Report
Ranek ?!? cmon man
BEST (most successful) mapmakers: 11/14/2014 23:53:48


Bananashake 
Level 60
Report
How can i not be on this list?!
I mean -> http://warlight.net/SinglePlayer?PreviewMap=34802

Edited 11/14/2014 23:57:40
BEST (most successful) mapmakers: 11/15/2014 00:06:01


Ranek
Level 55
Report
Inca ?!? cmon man Im just kidding. =) although you will always be my personal number one ...
BEST (most successful) mapmakers: 11/15/2014 00:10:36

qwed117
Level 49
Report
@Bananashake
I thought that was rejected!
BEST (most successful) mapmakers: 11/15/2014 08:50:16


Belgian Gentleman 
Level 56
Report
We have a great mathematican here in our Warlight community! No wonder he's the founder of a statistic clan.
BEST (most successful) mapmakers: 11/15/2014 10:22:54


Incaman
Level 56
Report
I'm not talking about his mathematics, I'm sure they are impeccable :). But when you attempt to value someones work you have 2 ways to do it. Either take in consideration his greatest achievement or you take into consideration all of his achievements.

Let me first state, so there are no bad conclusions, that RA doesn't have any maps rated lower then 3.5 and yes he makes better maps then me :D as do many others :D. Also ratings can be manipulated and subjected to personal bias (Qi's maps should be rated way higher and it's beyond me to understand why they are not), so they are not important.

So, if X made 10 maps and they are rated

8 - 2.3
1 - 4.3
1 - 4.6

you cannot just selectively choose the rating and say i will rate him only by the last 2 maps so 4.3 + 4.6 = 8.9 / 2 = 4.45

you either take into consideration all of his maps (2.3*8) + 4.3 + 4.6 = 27.3 / 10 = 2.73

or you just take into consideration his best rated map, hence 4.6

Forgive the rudimentary mathematics Math Wolf :D
BEST (most successful) mapmakers: 11/15/2014 12:03:58


Math Wolf 
Level 63
Report
@ Clement Atlee: "Skunk is only 0.031 points ahead of me" ... "I've only ever made 5."

You were one of the few mapmakers ranked below 50, who got 4 maps rated over 3.5, so I thought you deserved an honourable mention. Note that skunk still has 7 maps rated above 3.5, which is still quite a feat.



@ Phulesdorp: "So... from what I'm understanding, it's practically a Simpson's paradox to all "unpopular" maps?"

I don't really see the Simpson's paradox itself playing here? Although there is indeed a correction for number of times rated.
Technically, you could interpret it as one for example if two users each have two maps, one often rated, the other not often rated.
It may be that user 1's average rating is higher than user 2's average rating and user 1's best map is better than user 2's best map as well as user 1's 2nd best map better than user 2's second best map. Yet if user 2's best map is rated a lot, while user 1's second best map is the one rated a lot, user 2 may get a better rating in this case. (underlying explanation: user 1's best map may be overrated, or is not accurately rated).



@ Phulesdorp: "And will this list ever get expanded upon in the future"

If I have the time, I will repeat it in the future. If more information can be obtained in an efficient way, this may be added. The 'efficient' part is the problem here.



@ ChrisCMU: "I'd like to know where I rank. I think I only have 4 maps done, but all are well made IMO."

Your score and rank were added. I was surprised it wasn't higher, but the reason is because the two qualified maps both are sparsely rated. I think in your case, they are actually underrated (certainly the Great Lakes Basin which I will rate a 5 myself once my first game on it has finished), so getting more people to play and rate will most likely drastically increase your score and rank.



@ ChrisCMU: "I find this fascinating. Out of the top 50, I probably didn't even know 1/2 of those people made a map at all. It means either the rating system is flawed or they have great maps I need to check out."

I was equally fascinated. I didn't even know more than 70% of all mapmakers and I discovered some really nice maps that I want to start playing in the future. Some maps and mapmakers are probably overrated, but I cannot correct for alt-voting. Some diplomacy maps are highly rated probably thanks to diplomacy players and therefore not know by strategic players (and the other way around?) Also, older maps and mapmakers get a slight advantage because they are more rated. I may try to add a correction for this in a next version, but it requires even more data to be gathered and thus even more time to invest.



@ Bananashake: "How can i not be on this list?!"

You were ranked 163rd! The map you linked sadly did not qualify however. I personally have absolutely no idea why such an amazing, beautiful and strategically challenging map ideal for a 40 players FFA could not get a high rating, but maybe qwed117's reply gives a first lead towards a possible explanation. :-)



@ Incaman: "But when you attempt to value someones work you have 2 ways to do it. Either take in consideration his greatest achievement or you take into consideration all of his achievements."

I must strongly disagree with you.
* Taking only the best result doesn't take into account that this person may have many similar good projects. In this case, it wouldn't matter than RA made more than one map, nobody cares about those, only his best one?
* Using all maps penalises map makers who have more, but some not very high quality maps. It doesn't matter you have the 5 best rated maps of Warlight, you once made that duel map that everybody hated and now your average rating gets penalised for it and that guy who made one map, the sixth rated map, consequently is ranked first!
* My system tends to add value for extra quality while not penalising any lower rated maps that someone made. There may be other systems for that, but the two options you propose both make no sense.

Edited 11/15/2014 12:53:18
BEST (most successful) mapmakers: 11/15/2014 12:53:47


Incaman
Level 56
Report
"My system tends to add value for extra quality while not penalizing any lower rated maps that someone made. There may be other systems for that, but the two options you propose both make no sense."

Nope! You system tend to reward fluke success while not taking into consideration overall performance. In your system a guy who has made 4 maps rated 4.3 is worse than a guy who has made 10 maps rated below 2.0 and one 4.4. This doesn't mean that it is bad, it just means you compare one persons best performance against other persons best performance, while ignoring low performance.

And again: I don't want to say that anyone on that map doesn't belong there or that his maps are not good. I have the utmost respect for every mapmaker and also I'm very grateful for their hard work ( this of course excludes all those lottery maps and their makers )

"Using all maps penalises map makers who have more, but some not very high quality maps.

And not using them penalizes mapmaker who have been consistently making good maps.

It doesn't matter you have the 5 best rated maps of Warlight, you once made that duel map that everybody hated and now your average rating gets penalised for it and that guy who made one map, the sixth rated map, consequently is ranked first!"

Then if you care about rating, you shouldn't have made it. Maybe you should then consider taking into consideration only mapmakers who have at least 4 or more maps, i have no idea you are the mathematician :D. But it just doesn't work this way.

p.s. It just occurred to me while writing this, that this discussion we are having has no point. I don't care about your lists ( or anyone's for that matter ) and in the end it's your rating system do with it what you will :D.

I withdraw all my previous comments and well done it must have been hard work to calculate all that considering Warlights terrible map display.

Edited 11/15/2014 13:05:10
BEST (most successful) mapmakers: 11/15/2014 12:59:15


Bane 
Level 60
Report
certainly the Great Lakes Basin which I will rate a 5 myself once my first game on it has finished

im playing a multi on it now, and it is indeed awesome
BEST (most successful) mapmakers: 11/15/2014 13:10:24


Incaman
Level 56
Report
Does anyone know what map Nathan made?
Posts 1 - 30 of 79   1  2  3  Next >>