Hi everyone,

The following link is a graph I put together that has come out of a much deeper analysis of Warzone's 1v1 Ladder and how it rates/ranks players.

https://imgur.com/gallery/0FJ417i


The graph shows the average number of "games per interval" by the best rank obtained by players (a player in bucket "top 25" would be a player whose best rank ever achieved is between 11 and 25 inclusive).

Games per interval is a measure of how many games a player played before taking time off from (leaving) the ladder. Hence, average games per interval is representative to a large degree of players' attempts to put together separate "ladder runs" that allow for players ratings to not be effected by their previous games, often allowing them to obtain higher ranks as opposed to long continuous runs.

One key takeaway to highlight from this graph is that a lot of the attempts to game the Warzone ladder rating system (and its very generously short expiration window) are concentrated among players whose best rank ever obtained is in the top 25 (but worse than 10th). Those players average 14 games per interval. The next lowest group (excluding players never ranked) averages 26 games per interval.

I find it important in my data rating to distinguish between facts and opinions/hypotheses which have not yet been confirmed. Everything I have presented prior to this sentence is fact, the data confirms it, what follows is simply a hypothesis.

I think there are two important conclusions/suggestions that might be drawn from the wildly low average games per interval of players whose best rank ever achieved is 11-25.

The first is that when personally evaluating the skill of players, we should devalue players whose best rank obtained is 11-25, they are more likely to have gamed the system to get there (although in no way is this true of all, or even most of these players, it is simply more prevalent amongst them than the rest).

The second is that one way Warzone might consider combatting people cheating its rating system changing what statistics it displays on player profiles. Instead displaying players lifetime average ratings/ranks would kill incentive to try to make these types of runs given each individual run is just as likely to end in catastrophic failure as meteoric success, removing the incentive to attempt these types of runs.

On a final and slightly unrelated note, This graph and analysis is a very small piece of a much larger set of research I have been conducting on how Warzone's 1v1 ladder rates players. I will in the semi-near future be posting a series of blog posts detailing more of my findings. However, if there are any specific questions you would like me to explore feel free to suggest them and I will consider taking them on.

Thanks for reading,
Nick

Edited 1/17/2019 18:58:51

First. Very Cool. Actually super cool post :)

The first is that when personally evaluating the skill of players, we should devalue players whose best rank obtained is 11-25, they are more likely to have gamed the system to get there (although in no way is this true of all, or even most of these players, it is simply more prevalent amongst them than the rest).

Hmm, not sure about this. The first thing I thought when I saw the graph, is that a failed/short run tends to end in rank 11-25. Basically, to get a rank of 11-25 you don't even need 20+ games. I don't think this means that they are more or less likely to have gamed the system then other categories.

The second is that one way Warzone might consider combatting people cheating its rating system changing what statistics it displays on player profiles. Instead displaying players lifetime average ratings/ranks would kill incentive to try to make these types of runs given each individual run is just as likely to end in catastrophic failure as meteoric success, removing the incentive to attempt these types of runs.

It's the trophy that matters for most. You can hide ladder stats on profile already anyway so I don't think this change will have an impact tbh. A more popular suggestion is to change the rating system to something else, like Glicko 2.

I am looking forward to those blog posts!

Hmm, not sure about this. The first thing I thought when I saw the graph, is that a failed/short run tends to end in rank 11-25. Basically, to get a rank of 11-25 you don't even need 20+ games. I don't think this means that they are more or less likely to have gamed the system then other categories.

While my conclusion is certainly up for debate, I think you might be misinterpreting the graph a bit with your interpretation. The average games per interval is calculated by player. Essentially, the average number of games per run by players whose best rank ever obtained is in the given bucket. It is in fact usually not possible to receive a rank at all in a run (interval) of only 14 games, so this is likely an indicator that players whose best rank ever obtained is in the top 25 are bailing on runs before receiving a rank far more frequently than others (in many cases possibly because the run was not going to result in a good rating/rank.

An example of how this statistic is calculated would be as follows:

player A has recorded runs of 10, 20, and 30 games, his best rank ever obtained was 43.

player B has recorded runs of 18, 29, and 40 games, his best rank ever obtained was 9.

player C has recorded runs of 22, 31, and 30 games, his best rank ever obtained was 18.

player D has recorded runs of 20, 40, and 45 games, his best rank ever obtained was 12.

The Average Games per Interval would be...
Top 10: 29
Top 25: 31.3
Top 50: 20

Oh I see. I did miss the by player part.

It is in fact usually not possible to receive a rank at all in a run (interval) of only 14 games

Well, they still have a rank/rating per say. It's just that warzone hides them from the leaderbord/ official rank :P Either way I understand what you have done to create that graph now.

Hmm-time. I am curious to see how it looks if you zoomed in on top 25, with a box-plot for every rank. Hopefully you did this in a way where that is easy to do.

Really, really cool.
Thank you for the graph and your interpretation.
I'm definetely hyped and looking forward to read more ladder analysis.

However, I have another point to think about.
I realized that there are different Warzone personalities. Players who rank about rank 50+ (rating <1800) are more interested in just playing the game instead of winning. They don't care too much, they don't calculate properly, but they enjoy the strategic gaming anyways, because beating up some noobs in Quickmatches or AI's in single-player can be kinda boring.
Players who play better, let's say >1950 rating, care a bit more about actually winning the game. They get burnt-out much faster from thining minutes and hours about every turn, so when they lose 2 or 3 games in a row, they might leave the ladder - ragequit-like. They probably never had the intention of gaming the ladder, but still they finish their run with just 10 games played.

Another point: The strenght of a player changes over time. For example, I probably had many many "runs" in my warlife, many of them didn't end in the top 10, but for your analysis, you see me as a top 10 player.

Question: How do you measure when a run ends? No finished game for, let's say, 4 weeks?

Very nice analysis Nick, thank you for sharing.

I must admit, I have always had a softspot in my heart for boxplots.

Keying off of AI's comment

They get burnt-out much faster from thining minutes and hours about every turn, so when they lose 2 or 3 games in a row, they might leave the ladder - ragequit-like. They probably never had the intention of gaming the ladder, but still they finish their run with just 10 games played.

I preface this with noting my top 1v1 rank is 50.

I get the impression that there is a significant number of players (not necessarily a large group) who hop on ladders, take a run at the trophy, and then drop off the ladder either after getting the trophy, or after being discouraged from continuing to pursue the trophy. I am quite sure this is true.

Having said that, consider the psychology of these players, and, in particular, how quickly they leave the ladder. If a player quickly enters the top 10, that player likely will pursue the trophy for a while, likely playing quite a few games until they either attain the trophy or are truly discouraged.. However if that player peaks out at, say 20, and then starts struggling to win consistently or to move up in ranking, a ragequit becomes much more likely.

With this psychology at play, one can easily see how a player who hits a ranking wall at 20 in much more likely to quit than a player who hits the same wall at 5. With that, these players likely play far less games in their "run" than other players.

Edited 1/18/2019 15:56:31

Is a "run" considered to be average unexpired games, or the number of games played before leaving the ladder (including expired games)?

Edited 1/17/2019 22:04:37

" If a player quickly enters the top 10, that player likely will pursue the trophy for a while, likely playing quite a few games until they either attain the trophy or are truly discouraged.. However if that player peaks out at, say 20, and then starts struggling to win consistently or to move up in ranking, a ragequit becomes much more likely. "

I disagree with this sentiment. It's the people who hit rank 1-10 who I'd consider more prone to ragequit. The 1v1 ladder penalizes losses far more than it rewards wins. The vast majority of people that get rank 1 debut at rank 1. I'm very capable of getting in the top 10, but if my goal were to join and get rank 1 I'd quit after 1 loss unless it was vs. someone over 1900. Even at 19-1 start you only get rated around 2200 typically.

Climbing from 2200 to 2400 to take rank 1 is exceptionally hard once you have losses.

As an example, Rufus is rated 2387. That's a high rating, sure, but if he was 35-0 instead of 33-2 he'd be at 2553. That's a big difference. The first couple losses someone gets tanks their rating hard from what it was when undefeated, and to raise it back to that previous level requires an insane amount of wins against highly rated opponents.

This is why if I were to join the ladder today and aim for rank 1 I'd quit at 20 games if I debuted at 2200-2300. If I debuted under 2300 my only chance to hit rank 1 would be to wait for Rufus to leave, to beat him head to head, or to wait for him to lose more games.

Edited 1/18/2019 00:30:19

Final conclusions aside. Great job Nick, I love the information!

@The Last Kiss

Thank you for responding to my comment. I wasn't aware of the interactions between W-L and rating at the top of the ladder, so I appreciate what you have added. Let me expand a little further given what you said.

" If a player quickly enters the top 10, that player likely will pursue the trophy for a while, likely playing quite a few games until they either attain the trophy or are truly discouraged.. However if that player peaks out at, say 20, and then starts struggling to win consistently or to move up in ranking, a ragequit becomes much more likely. "

I disagree with this sentiment. It's the people who hit rank 1-10 who I'd consider more prone to ragequit. The 1v1 ladder penalizes losses far more than it rewards wins. The vast majority of people that get rank 1 debut at rank 1. I'm very capable of getting in the top 10, but if my goal were to join and get rank 1 I'd quit after 1 loss unless it was vs. someone over 1900. Even at 19-1 start you only get rated around 2200 typically.

Climbing from 2200 to 2400 to take rank 1 is exceptionally hard once you have losses.

As an example, Rufus is rated 2387. That's a high rating, sure, but if he was 35-0 instead of 33-2 he'd be at 2553.

Alright, now keying of your view that if you were pursuing the trophy and you lost to a sub-1900 player, that you would likely leave the ladder, what is the likelihood of you losing to a sub 1900 player? On the other hand if a player debuts ranked 20th to 25th, their likelihood of losing to a sub-1900 player is quite high (assuming of course their debut ranking and rating is representative of their true skill level). In their next 10 games, they are quite likely to lose several games.

These players, per the original assumption, got on the ladder in hopes of reaching number 1, or at least top 5, but by their 10th or 20th game, they will realize that they likely can not break into the top 10. Then they leave the ladder.

Is this ragequitting? Maybe discouraged is a better word?

On the other hand, your point is quite valid that a more talented player who could reach top 5 might by error or bad luck drop a game to a lower ranked player quite early. In that case, as you described, the talented player might just quit given how difficult it will be to reach number 1.

In Warzone, it is always possible to lose to a <1900 player. That is because this game involves a little bit of randomness and luck, whereas in chess for example Magnus Carlsen would never lose a single match against a <2300.
And that's alright. The problem is, that Warzone players get punished as hard for this loss as Carlsen would get punished.

Note that plenty of failed ladder runs don't ever finish enough games to get a rating, so that can skew the results a bit.

Note that plenty of failed ladder runs don't ever finish enough games to get a rating, so that can skew the results a bit.

They don't finish Fizzer's requirement to be shown on the leaderboards. However, if you play one game or more, you have a non-zero rating. It might be ridiculously inaccurate at one game played, but it's still there :)

Is a "run" considered to be average unexpired games, or the number of games played before leaving the ladder (including expired games)?

While it would have been great to be able to calculate runs by measuring when players left and rejoined the ladder this was unfortunately not possible given the available data. Instead, for each player, their average number of games per interval ("run")was calculated in the following manner.

Players receive a rating when they have at least one unexpired game. So by simply looking to see when a player's rating is unavailable you can see when they have let all of their previous games expire, and we call this a run. It is by no means a perfect method but it is pretty accurate, and certainly accurate enough to give a good picture.

"Alright, now keying of your view that if you were pursuing the trophy and you lost to a sub-1900 player, that you would likely leave the ladder, what is the likelihood of you losing to a sub 1900 player?"

Not that unlikely. (The Last Kiss is my alt, most people know that, but just clarifying that in case someone doesn't.)

I started 19-1 in 2017, with my only loss being a brawl game where I made a very minor error vs a player rated in the 1700's. That particular player I'd anticipate beating over 90%, but it was a very brawly map where there was a very obvious set of "best picks" that resulted in he and I making the exact same picks in the exact same order which hugely reduced the margin of error allowed in the game.

At 19-1 I got close to 2200, after winning 3 more games to be 22-1 I reached 2214, if I had been 23-0 instead however I'd have hit 2420, that one loss is the singular reason I don't have the 1v1 ladder trophy.

I think there's actually a lot of people that lose more to low rated opponents as well due to them being harder to predict at times. I think it's disingenuous to some degree to blame opponents for ones defeats, but there is some truth to it that it is harder to predict people who don't play the game at the same level you do.


Example: From the run where I nearly hit rank 1:

Ranked 62nd with a rating of 1709. adiba's profile
Best rating ever achieved: 1816. Best rank ever achieved: 50th

That player defeated me, yet I had wins vs all of the following in the first 23 games:

Ranked 50th with a rating of 1740. Sebus Maximus's profile
Best rating ever achieved: 1863. Best rank ever achieved: 34th

Not Ranked with a rating of 0. Lewis's profile
Best rating ever achieved: 1956. Best rank ever achieved: 17th

Not Ranked with a rating of 1905. sloppyfatginger's profile
Best rating ever achieved: 2078. Best rank ever achieved: 6th

Not Ranked with a rating of 0. Octane's profile
Best rating ever achieved: 2123. Best rank ever achieved: 7th

Ranked 7th with a rating of 2111. AI's profile
Best rating ever achieved: 2136. Best rank ever achieved: 5th

Not Ranked with a rating of 0. Boubou's profile
Best rating ever achieved: 2016. Best rank ever achieved: 11th

Not Ranked with a rating of 2062. Turkish's profile
Best rating ever achieved: 2146. Best rank ever achieved: 6th

It was my 8th highest rated opponent that delivered me my first loss.

My 2nd loss again came at the hands of someone I usually consistently beat:

Not Ranked with a rating of 1813. Bigchps147's profile
Best rating ever achieved: 2003. Best rank ever achieved: 14th

Bigchps is good enough to beat anyone, all three of his notable wins are vs malakkan for example, so I'm far from the only player he pulls upset wins against, but historically I have beat him far more often than I have lost to him.

Similarly, Rufus currently has two losses:

SovereignJHG50 defeated Rufus 17476430 1/8/2019 14:51:37 SovereignJHG50: 1770

RynoFury defeated Rufus 16987812 11/16/2018 11:35:08 RynoFury: 1948

Those are both pretty big upsets. Not saying those players are bad, but they're far from the best opponents he's faced thus far considering the following all peaked over 2000 or are over 2000 currently:

Rufus defeated Kenraali 16922092 11/12/2018 13:54:38 Kenraali: 1019 (Ignore current rating, Kenraali had peaked at 2043, boots are the cause of his current rating.)
Rufus defeated AI 16969351 11/16/2018 07:47:37 AI: 2111
Rufus defeated Pardon99 16952062 11/20/2018 16:16:48 Pardon99: 1932 (Peak rating 2352, few unexpired games leading to a low current rating)
Rufus defeated fireice82 17066702 11/27/2018 09:55:23 fireice82: 1714 (Peak rating 2081.)
Rufus defeated malakkan 17066701 11/27/2018 11:52:43 malakkan: 2321
Rufus defeated Milly 17154301 12/4/2018 10:19:34 Milly: 1934
Rufus defeated AlturoSensei 17127976 11/30/2018 14:31:22 AlturoSensei: 2123
Rufus defeated Carlitos612 17098123 11/30/2018 11:55:07 Carlitos612: 1664 (Peak rating 2085.)
Rufus defeated Bewmaster 17213241 12/9/2018 12:26:41 Bewmaster: 2137
Rufus defeated Kurdistan49 17437937 1/2/2019 09:08:45 Kurdistan49: 2196
Rufus defeated Super Smoove 17380883 12/29/2018 09:28:19 Super Smoove: 2200
Rufus defeated 89thlap 17277834 12/22/2018 17:57:05 89thlap: 2181
Rufus defeated Bbraw 17237383 12/14/2018 06:13:40 Bbraw: 2086
Rufus defeated jimmy 17502327 1/18/2019 11:52:44 jimmy: 2148

That's 14-0 vs people rated over 2000, with 2 losses to sub 2000's.

Edited 1/21/2019 10:59:25

At 19-1 I got close to 2200, after winning 3 more games to be 22-1 I reached 2214, if I had been 23-0 instead however I'd have hit 2420, that one loss is the singular reason I don't have the 1v1 ladder trophy.

you can ladder again tho and this time go 23-0 and win the trophy. :D

Alright, speculation aside I am going to mathematically give everyone the answer to the question about losing to sub 1900 players based on Warzone's current rating system. For anyone who doesn't know, Elo rating system's have a few nice formulas involved with them one of them predicts the win probabilities of 2 players facing each other. Keep in mind that since this rating system is not perfect neither are the predictions, but this should give you a decent idea.

player_a_win_prob = 1.0/(10.0^((team_b - team_a)/400.0) + 1.0)
player_b_win_prob = 1.0 - team_a_win_prob

So, as an example, let's take the probability of a 2200 rated player losing to a 1700 rated player

p(2200 rated player beats 1700 rated player) = 1/(10^(-500/400)+1) = .9468

So, while it is highly unlikely in theory that a 2200 rated player loses to a 1700 rated (about 5% chance), it is far from unfathomable.

Use this formula to test out any other matchups you might be curious about :)

Edited 1/22/2019 15:05:18

@Nick: Are you able to make a graph which shows the actual outcome compared to the expected outcome related to the formula you used before?

My prediction is, that for the example 2200 vs 1700 the actual score is not 5% (as predicted) but something like 10%. Same for other cases then.

Can you do that?

There's a few problems with the ELO calculations in this particular situation:

1) ELO assumes a logistic distribution of skill, which may or may not be the case on Warlight.

2) Warlight uses Bayesian ELO, which can highly overestimate ratings with a low sample of games. My highest rating is 2340; i definitely do not have a winrate of 94.68% against 1800-rated players. So when you try to compare a player with a rating of x and another player with a rating of y, you have to make sure they're at least in a certain confidence interval of actually having that rating. Which is something that Bayesian ELO fails to do a lot.

Other than that, really interesting data!