Everyone except berdan131 is right. Thank you so much for explaining the issues in such a detailed way, Farah!

oh fizzer, I pray for the day
when Bayesian ELO goes away,
ladder runs are a distant memory,
and players have to prove their supremacy!

Fizzer, please change Bayesian ELO to regular ELO!

IMO the guys being passionate here miss that guys have been passionate about that topic even before some of the players here (who lied when they were asked whether they are above 13) have even been born. Here is what will happen: You guys will retire and in a couple of years the next WarLight generation will discover the ELO system being flawed.


https://www.warzone.com/Forum/1251-vote-ladders-switch-traditional-elo-model?Offset=30

The polling period has ended - thanks to everyone who voted. Time for results!

Lots of people are definitely passionate about this issue, and a lot of good points and ideas have been brought up in this thread.

I was a bit surprised by the results:

- 10% of voters chose to switch to Traditional ELO
- 43% of voters chose to stick with the current system
- 37% of voters would like to pursue some other rating system
- 10% of voters chose the "I don't care" option

The first conclusion I can draw from this is clear: Traditional ELO is out.

Let's look at the rating system options:

- WHR: After reading the introduction on this paper, it certainly sounds better than the current Bayesian system. However, no implementations are available, which makes this one of the most time-consuming systems to adopt. If someone released an implementation it would make this doable, but for the moment it's not really feasible without a large investment that I don't have time for.

- TrueSkill: [Implementations](http://www.moserware.com/2010/03/computing-your-skill.html) exist, so TrueSkill is more feasible. More investigation needs to be done in order to determine if it's really the best fit and how much work it would be to integrate.

- Bayesian ELO (current system): While certainly not perfect, it has two things going for it:
- The implementation cost is zero, since it's already implemented.

- More WarLight players prefer it than any other system. Potentially even by a wide margin, since the 37% that chose they wanted to pursue some other rating system may be fragmented between multiple different rating systems.


It's important to understand that the most limiting factor of what can be done with WarLight is development time. I have to carefully choose what I spend time implementing, as there is easily over a thousand different things I'd like to be adding to WarLight right now. What these poll results are telling me is that changing the rating system shouldn't be at the top of that list right now.

I don't want to change for the sake of change. Yes, the current system has warts, but every system will likely have warts. It can take a lot of investigation to determine which warts are preferrable.

I know some people won't be pleased with these results. This doesn't mean that the rating system will never change, but it does mean that I'm not going to consider it my highest priority at the moment.

If anyone has development skills and is super passionate about getting a different rating system implemented, I'd welcome the help in investigating a new one - send me an e-mail!

@alexclusive

My posts were right. Fizzer said exactly what I assumed he would think.

Fizzer: "The implementation cost is zero, since it's already implemented [Bayesian]. The most limiting factor of what can be done with WarLight is development time. I have to carefully choose what I spend time implementing."

Berdan: "Maybe it's too much work to bother and fizzer doesn't see the value added is enough to justify it."


Of course, I didn't know what Fizzer said 9 years prior. However, my guesses were 100% accurate. How is this possible? Call me a genius. I just see more than other people because my brain is very highly developed. It's that simple.

We all know that Fizzer's development time is the limiting factor. This is quite trivial and obvious.

The point is that this change in the rating system has a good ROI when you consider the benefits for the community and the development time.
It is unfortunately quite frustrating to see how this topic is ignored by Fizzer over and over again and instead more complex features are implemented that nobody asked for and whose benefits are completely obscure. Take the commerce mode as an example: It was certainly elaborate and I wouldn't know that anyone really wanted it. At the moment it is practically not used at all.

Of course Fizzer is free to decide what to implement and prioritize. But there are only limited possibilities for us users to suggest and prioritize our own things. The uservoice-tool is ignored and suggestions in the forum are mostly ignored.

The new rating system is a hot topic and many people are annoyed by it, even if Norman is now digging out a 9 year old survey, which of course does not reflect the current opinion.

Edited 9/23/2020 07:39:54

one thing that's changed in the last 9 years:

all the repeated drama from ladder runs,

that's been enough to cause many players to realize that bayesian elo has a big problem.

It is unfortunately quite frustrating to see how this topic is ignored by Fizzer over and over again

From what I understand, it is not ignored and rather a matter of priorities.

People who are on this website for a while, should realise by now that Fizzer prefers to communicate about features when they are almost ready to be rolled out, maybe to avoid high expectations, community pressure and impatience? Either how, we can expect that any update, if it happens, will be announced only a few days before it actually goes through.

I firmly believe that there will be updates for the ladders again at one point, just as it took quite a while to have multi-player ladders added, and he also added an RT-ladder by popular request. Personally, I do know that he is aware of the issue and willing to improve it at some point, because it has been communicated to him by multiple people, including me, and at varying moments. We'll just have to be patient and respect his timeline and when and how such an update will happen.

In the meanwhile, nobody should underestimate the input that the community has indirectly. Civil discussion with easy to understand arguments, nicely illustrated by Farah in creating this topic, can inform Fizzer about the best course of action to take.
Opposite to that, blanketing the forum with multiple posts just repeating your same stance over and over, or purposely breaking the rules to prove a point, might arguably be less effective.

Pro-tip: smart people will show their smartness with content as in Farah's first post, not with trolling or irrelevant hindsight and selection bias. (Which is not a personal dig at berdan, but also at modern types of media figures and politicians who copy this type of behaviour, formerly only shown by internet troll and at the local neighbourhood gossiping, and now at a much more dangerous scale.)

"1. Win game lose rating.
Hmm, is it that common? Usually you gain rating."

It's only common for people who have few games IMO.

Nauzhror vs Kevin Turner 23929502 Kevin Turner: 1553

I'm going to lose 2 rating when this game ends. The ladder basically doesn't think I should be playing him, and it's right. I'm 2155 rating at the moment, he's 1553, after I beat him I will drop to 2153. The thing is, the ladder rarely matches people with 600+ separation, it in fact didn't here either.

I had 1552 rating when this game started, thus at the time it was a perfectly valid matchup, but I rose 600+ rating since then. This is only a common thing for people with few unexpired games, who as a result have volatile and wildly fluctuating ratings. If my rating was cemented it'd never fluctuate enough to result in me ending a game 600+ elo higher than I began the game, and as such wouldn't result in me beating someone 600+ elo below myself.

This is an issue with the idea of concurrent games. In most activities a game starts, and then finishes, you can't play 5 games at once, thus the elo you start a match with, is the elo you end it with (before adjustments caused by the game of course).

Edited 9/23/2020 08:52:28

Competitive is for old and senile Europeans.

@quicksilver:
one thing that's changed in the last 9 years:

all the repeated drama from ladder runs,

that's been enough to cause many players to realize that bayesian elo has a big problem.

As I have said, there was a competitive scene in the days way before you joined. People knew about the flawed ladder system at least since Doushibag dethroned The Impaller way back in 2011.

People were way more blatantly gaming the system in the past than nowadays since now we have guys like MoD and also the general community heavily shaming the stallers.

Hey, happy one month anniversary. It's been two days, 23 hours on this move, figured I'd catch you here when you show up in forty minutes or so. It's been a lot of fun. Because of guys like you, i don't get to play on the 1v1 ladders. That's right, this is just one of the games people are stalling against me, and by far the most eggregious. I've got four games running right now that AVERAGE over 2 days, 12 hours since their last move. So I get to make a move every three days in these games while you guys steal the $30 I paid to play on this site. Have "fun" with your "strategy" - although I can't see how logging in at 5am to make a move every three days in a losing effort is "fun" and I can't see how you can be proud of this "strategy". This seriously sucks, and I really wish you'd stop.
https://www.warzone.com/MultiPlayer?GameID=1229199

since now we have guys like MoD and also the general community heavily shaming the stallers.

... this is exactly the problem - makes life much much worse as there's lots of shaming / toxicity / ganging up on people / intense fighting

Edited 9/23/2020 13:30:37

lol Norman, I remember those early days, thank you for the reminder!

Indeed, some of the old strat community (among others Doushibag, Duke, Impaller, Teddy, BP, crafty, and a newbie called Math Wolf) quickly developed an insight in how the system worked. The reason for the poll and ensuing discussion was that Doushibag, among several others, were already illustrating the flaws of it for this setting in the most eggregious way. (And that was before expiration, as the ladder was at that time less than 3 months old! (the time for games to expire in the early days)

That game was March 2011, only about a month after the 1v1 ladder and memberships were introduced.
(Easy to remember and check as I joined before the ladders and I bought my membership because of it as only members could join at that time). Stalling and problems with BayesElo are effectively as old as the ladder. I actually had forgotten that my proposed solution (a decay function linked to activity), applied in a modified shape in MDL, is apparently as well.

For anyone interested in this topic, I'm currently looking into different rating system and trying to create one that could be used by Warzone. I've created a discord server where discussion can take place: https://discord.gg/yD79E8

just as it took quite a while to have multi-player ladders added,

The ladders used to be single player? How does that work?

Great post Farah.

Here in the USSR we use a Elo rating for our tournaments based on chess. We are open for discussions or better elo methods.