<< Back to Off-topic Forum   Search

Posts 1 - 8 of 8   
MDL Analysis: 12/11/2017 03:53:10


knyte 
Level 58
Report
TL;DR: I did some stuff with some data from the MDL. There's a bunch of it, so I'll go over it over multiple days. If you want to see all of it now, it's at http://bit.ly/mdl-analysis. Ctrl+F "Parts List" for list of parts.

A couple years ago (https://www.warzone.com/Forum/95457-makes-template-strategic), I was really interested in quantifying how strategic a template is. Came up with some stats, actively ran those numbers on IPL (http://tinyurl.com/infinity-premier) and the official ladders, but the project was quietly abandoned since there wasn't a great source of data.

But now, thanks to MotD and Muli, we've got a multi-template Multi-Day Ladder (http://md-ladder.cloudapp.net/) with over 10,000 finished games on 42 active templates (and 9 retired) played by ~360 players (~140 of which played long enough to get ranked). That's a treasure trove of data to work with- probably about the best anyone could've expected by now.

That really ticked me off- so I tried to crash the Multi-Day Ladder by getting data on as many games as it'd let me in order to overload its RAM and cause its Visual Basic GUI to backtrace MotD's IP address and make the CPU overclock itself until it exploded. Unfortunately, that plan did not succeed but I did get some data to work with.

I wanted to finally get some numbers to make those template rankings, but using just data for player ratings and about 3500 game results, I played around and did a little bit more. It would be a long post, even by my standards, if I decided to just cover everything at once, plus if I space things out there'll be more room for back-and-forth fun. Keep in mind that this isn't meant to be rigorous in any way (hold this to the same standards you would a friend's amateur World Cup predictions), and in fact there's some glaring issues/limitations on everything I did that I'll cover as we go along. This was done solely for entertainment, and so I opted for the least pain-in-the-ass approach at every turn. Don't expect any of this to be novel, interesting, unique, or worthwhile. So, without further typing, here's everything I have so far:

Published Google Sheets workbook - http://bit.ly/mdl-analysis (raw: http://bit.ly/mdl-analysis-sheet; charts: http://bit.ly/mdl-analysis-album)
Python code for gathering the data - http://bit.ly/mdl-analysis-code

Most of the real work happened in the Google Sheets workbook, and if you create your own copy you'll get all the data anyway. If you plan on adapting my Python code, you should probably message me (as there's some bugs/limitations you should know about) and hit up MotD if you plan on hitting the MDL servers.

I'm going to be sharing/commenting on these analyses in 6 (planned) parts. I'll keep linking to them in this comment for convenience.

Parts List
--------
Part 0 - Introduction + Activity: https://www.warzone.com/Forum/289534-mdl-analysis?Offset=2

Thanks for reading all that, and don't hesitate to post feedback/suggestions/ideas as this goes on. I'm only going to be using ratings/game result data (gathered last Saturday) and won't really go and do anything crazy since I've already done pretty much everything I planned to do with this data.

I'm not a statistician, and this isn't real statistics, so let's have some fun!

PS: if you want to help me with my mission of overloading the MDL servers (or just creating more data for future analyses using the MDL), you should join the best ladder on Warzone: http://md-ladder.cloudapp.net

PPS: Mods, please keep this in the Off-topic forum just for all the people who have it filtered out in Muli's userscript. ;)

Edited 12/30/2017 08:19:35
MDL Analysis: 12/11/2017 04:14:30

TBest 
Level 60
Report
PPS: Mods, please keep this in the Off-topic forum just for all the people who have it filtered out in Muli's userscript. ;) 
MDL Analysis: 12/11/2017 04:33:59


knyte 
Level 58
Report
Part 0 - Introduction + Activity

This part's gonna be short (2 graphs)- I'm basically just posting a little bit of content so I have an excuse to talk about some of the methodology. (Scroll to the bottom if you don't care).

To do this whole analysis, I sampled as many games I could from the MDL (4101), removed all the ones that were deleted (due to not being accepted), and only used games where both players had played enough to get ranked on the MDL (in order to trim inaccurate ratings). This came out to 3502 games played by 140 players.

For each of those 140 players, I also retrieved their ratings from their MDL profiles. I initially used their current ratings, but that broke due to something I'd like to call the "Muller-Chigurh Effect" (comparable to the "Boston Phenomenon" on the 1v1 ladder)- Lucien Muller (http://md-ladder.cloudapp.net/player?playerId=2854345641) and Anakin Chigurh (http://md-ladder.cloudapp.net/player?playerId=287161802) broke a lot of my analysis because of how poorly their current ratings effect their actual skill. I could've just removed these two from the analysis, but they weren't the only offenders- plus, if I could get reduce their data to just the accurate parts, then they're still useful. So I went by peak ratings since (despite their flaws) they're less inaccurate on the whole.

Except that broke too- see, if you're a bad player, your rating keeps going down quite a bit but your peak doesn't reflect that. I'll pick on Benjamin628 (http://md-ladder.cloudapp.net/player?playerId=9936698896) since he's retired, but if you check his profile you see that he's been declining ever since he hit his peak rating of 1333. If we go by peak rating, all the weak players are all going to cluster somewhere around the 1300-1450 range. So the ratings I use are your peak rating if you peaked over 1550, and your current rating otherwise. 1550 might seem arbitrary at first (hint: because it is) but the odds of a <1450 rated player hitting 1550 or higher at any point (even with some crazy unlikely matchups) are well below 5%.

So that's the ratings, and with those ratings, you can go to the analysis (http://bit.ly/mdl-analysis) and check some other stuff that came out of them. Under "Matrix" you can see how players are rated on different templates (along with their overall rating that I use), and under "Upsets" you can see the biggest upsets I was able to find (and the biggest beatdowns, if you scroll all the way to the bottom). You can also look under "Players" for more general player stats.

For benchmarking reasons, I also sampled about 1600 games from the 1v1 ladder + the corresponding ratings of ranked players. And now, onto the real substance.

If you're like me and keep tabs on all the failed competitors to Warzone, you'd have noticed that one of them- Art of War- apparently has a serious activity issue (https://www.artofwar.cc/forums/viewtopic.php?f=8&t=876&sid=0947949cd618a9f2460c096991e6bb41). It turns out that over 40% of their games are played by just the top 10 players.

So that got me wondering, do we see any hint of that in Warzone? There's clearly a participation gap on the forums and some ladders- some players are way more active than others, and 99% of Warzone is invisible. But within ladders, are the good players also just way more active than the rest?

Well since we have ratings + game results, we can just directly compare how good a player is and how many games they've played (among the ones I've sampled).

Multi-Day Ladder: Game Count vs. Rating
(interactive: https://docs.google.com/spreadsheets/d/e/2PACX-1vR1rBQasy4-hlhM1D-VJy7eP7IaE4qIw0wAZ81ho_qCFfUrAlABDtY7egV4pXEl9iJPucHgBeyYvvpB/pubchart?oid=980615800&format=interactive)


1v1 Ladder: Game Count vs. Rating
(interactive: https://docs.google.com/spreadsheets/d/e/2PACX-1vR1rBQasy4-hlhM1D-VJy7eP7IaE4qIw0wAZ81ho_qCFfUrAlABDtY7egV4pXEl9iJPucHgBeyYvvpB/pubchart?oid=229198557&format=interactive)


The r^2 is basically the percentage of players whose game count can be "explained" by their rating (using a polynomial regression here, which is near-identical to the linear one for the 1v1 ladder). Sidenote: r^2 doesn't actually work for non-linear regressions, but this one's close enough to linear and Google Sheets doesn't make it easy to retrieve standard error so ¯\_(ツ)_/¯

It looks like there is a bit of an increase in activity as players get better, but it's also not at the level where your skill level and your activity on either ladder is closely tied together (you can see a lot of noise). You can also see that the activity increase is a bit more pronounced on the MDL than the 1v1 Ladder- perhaps because you can play more games at once, or just because better players like the MDL more than the 1v1 Ladder.

You probably also see a couple of dots hovering real far above everyone else. Who are these two that need an intervention? You can click the interactive links to find out :) Since the font size is a bit small and some of the names still cover each other up, you can also just get this activity + rating data by looking at the Players tab in the analysis.

I'll just tell you, though. The most active player on the MDL is Muli (341/3502 games! He was involved in nearly 10% of the last 3500 games on the ladder!) and on the 1v1 Ladder it's KneelForZod (32/1100). KneelForZod, as Nauz would tell you, is one of many alts of the same Italian guy, and across all those accounts he might also be giving Muli a run for his money.

That's all for now! Feel free to chime in with ideas, corrections, or whether you think Bill Belichick is gonna stick around in New England after Brady retires (it's in Off-topic, after all). I promise you there's going to be less boring paragraphs in the next one- I just wanted to get the methodology-shaped elephant in the room out of the way.

Edited 12/30/2017 08:19:21
MDL Analysis: 12/11/2017 09:09:57


malakkan 
Level 63
Report
Hi Knyte,

I've checked your analysis spreadsheets and found your work quite novel, interesting, unique, and worthwhile. Although it obviously raises a lot of questions about your methodology and its limitations..
But I guess you've decided to put it in the off-topic forum precisely to avoid having to deal with comments from Math Wolf fandom , so I'll just say I'm looking forward to reading the coming posts..
MDL Analysis: 12/11/2017 14:54:48


master of desaster 
Level 65
Report
Looking forward for more data. I don't really agree with your methods (peak/non peak rating) but everyone can do with the results whatever he likes :)
MDL Analysis: 12/11/2017 16:22:56


kyte
Level 24
Report
Thanks for the support! Math Wolf/rigorous comments are 100% welcome. I just put this in OT to lighten the mood and lower the bar, since I was more worried that this analysis' results would get taken too seriously.

Edited 12/11/2017 16:23:53
MDL Analysis: 12/11/2017 23:12:18


linberson 
Level 61
Report
I would like to hire you.
MDL Analysis: 12/15/2017 04:28:13


knyte 
Level 58
Report
Note on the hiatus since Sunday: I'm typing up commentary in the charts album (http://bit.ly/mdl-analysis-album) that I'll abridge for future posts on this thread. I also forgot that I'm gonna be out of the country Saturday-Saturday so we might have to pick it up closer to Christmastime.
Posts 1 - 8 of 8