<< Back to Ladder Forum   Search

Posts 1 - 23 of 23   
Season II matchmaking: 3/29/2012 14:05:58

The Duke of Ben 
Level 55
Report
In playing Season II, I've noticed that matchmaking is extremely tight. Through the second half of my games, I found myself playing the people immediately above or below my own rank.

I was curious to see if the rest of you have any thoughts on the very close matchmaking.

Some pros:

Better matchups between equally skilled opponents.
Less chance for a really bad player to upset a really good player, or for a really good player to trash a bad player for no gain (Yeon problem from Season I).

I think this system did match me very well with players of around my own skill level, and therefore resulted in good matches for the most part.

Some cons:

Very limited mobility after the first few games. People who won their first games played high ranked players, while people who lost them played low rank players. For two equally skilled players, one of which won the first game and one of which lost, the player who lost will be at a serious disadvantage throughout the season. (For clarification if anyone doesn't know what I mean, keep in mind that a loss to the top player hurts your score a little bit, while a loss to a low ranked player hurts it a lot. Conversely, beating a high ranked player helps your score a lot, while beating a low ranked player may not help it at all. If you never play low ranked players, then you are never open to the risk of a huge loss, and if you never played high ranked players, you are never able to regain that position).

If you only play against players at your own rank, your rank will only change a little bit after each game. Recovering from one or more early losses, even to good players, may take the rest of the season. Taking a further loss, or losing to someone very low ranked (which happens even to the best players on occasion), may doom a strong player. Several players who were top 10-20 on the standard ladder got stuck at low rankings this season. If they managed to play someone higher than the rank 80-100 players they were next to, they would have moved up in ranks fairly quickly. Instead, they would gain so few points for a win that they were essentially stuck, even if they won all the games they played.

That was longer than I had planned. Any thoughts on the issues?
Season II matchmaking: 3/29/2012 14:07:50

The Duke of Ben 
Level 55
Report
For clarification, I think that Season II's matchmaking was far superior to Season I's. I don't think that it's quite good enough, but I wanted to open up the discussion to other people, and get more opinions.
Season II matchmaking: 3/29/2012 14:50:03


Ace Windu 
Level 56
Report
I had been meaning to make a thread on this :P Thank you for doing it.

I agree with everything you've said. The matchmaking is too tight IMO. It has led to a lot of very good games but as you said it froze out a lot of good players because of relatively poor starts. The only way to get up the ladder was to blaze through every game ASAP to get the ranking bonus (presuming you won ofc) before your next match-up.

The tight matchmaking has also led to people holding out on losses at the top of the ladder so that they get matched against the best players possible. Minimising the risk of losing points and maximising the opportunity to gain.

I was lucky to get a high rank early on and since then the lowest I've been was 11th. 3 players in the top 10 have never fallen outside of the top 10. Obviously that's to do with their play as well as with the matchmaking system but I do think the matchmaking this season has led to some degree of stagnancy in the rankings.

I would like to stress that I think this season has been/is great, but the matchmaking needs another tweak to make it more open IMO.

P.S. I wish alababi and V had continued with the season, they'd be making me look a lot better right now XD
Season II matchmaking: 3/29/2012 15:06:19

The Duke of Ben 
Level 55
Report
Hah, I also beat Alababi early on, and his rating tanked immediately after. Sadly, he managed to beat Dr. TypeSomething and the Dr. also got destroyed early on. Zaeban abandoning the ladder certainly didn't help any of the people he beat, but luckily that was later in the season. If he had beaten a couple of strong players and dropped out before game 5, he could have locked some people out of the top 10 for the rest of the season.
Season II matchmaking: 3/29/2012 15:12:31


Ace Windu 
Level 56
Report
Zaeban beat me lol thankfully he only dropped to around 50th
Season II matchmaking: 3/29/2012 15:58:10

Qi 
Level 55
Report
If each new season could partly base match-ups on the results of previous completed seasons, the game count per season could be reduced (to avoid being like Mek Blaze: one day he woke up with about 10 new games waiting to join against 10 of the best 1v1 players) and the first few games wouldn't matter as much.
Season II matchmaking: 3/29/2012 15:59:09

Qi 
Level 55
Report
*to be joined
Season II matchmaking: 3/29/2012 16:47:24

The Duke of Ben 
Level 55
Report
While that does reduce the importance of the first few games, it doesn't solve the long term problem of trying to advance in rank once you've fallen behind. Or, as a related but less serious problem, being unable to fall too far in rank because you never lose too many points at once.

Looks take the example of lobstrosity. He played 6 games in Season II, but lost his third game to a low ranked player, and fourth game to a higher ranked player. Assuming that he finished the last 14 games and won almost all of them, he still would likely be ranked lower than 30. (His actual rank is something like 103 right now). He would essentially have to fight his way all the way up the rankings by the time his 20 games were up. On the regular ladder he has all the time and games he needs to average out his win rate. For instance, on the regular ladder he lost to a guy with a rating of 1450ish shortly after joining the ladder. That didn't stop him from reaching #1, because he had far more than 20 games available in order to compensate. The regular ladder also allows for matchmaking much further from your own rank, which means a good player can beat good players and gain rank much faster.

By having the previous season help set up new season matchmaking, we would actually be making some of the problems worse. lobstrosity would be joining the new ladder with a ranking of over 100, meaning that he doesn't even get a chance to win the first few games and still compete for #1. In other words, despite being a #1 capable player, he would be locked out of contention for that spot based on his performance in the previous season.

The real culprit isn't so much the first few games, but the close matchmaking that does not allow someone to advance or fall too much, once rating has been established.

If my experience is universal, then everyone was being matched with people who were within 3 spots of themselves. If we establish an initial rating after game 10, then the number one player at game 10 could theoretically lose the next 10 games and still be ranked fairly high (having never played against a low ranking player).

The problem actually compounds itself since the top ranked players play other top ranked players, meaning that all of their ratings go up together. PaniX and Mek Blaze (for instance) boost each others ratings by having high ratings themselves, and playing each other. Then they each play Chris, who also has a high rating, and never risk a big loss of rating, win or lose they stay in the top 10.
Season II matchmaking: 3/29/2012 16:55:35

The Duke of Ben 
Level 55
Report
Waya, and regrettably, Yeon, are both good examples of the problem in action right now.

Waya lost to a mediocre player early on, then lost to zibik. Then Waya beat a lot of people, but because of low rating, they weren't worth a lot of points. Waya fought up to where they could get matched with Panix (#1), and beat PaniX. Despite this, there are not enough games left for Waya to reasonably take first place. Too many games were "wasted" playing low ranked players, so Waya cannot gain enough points. Had Waya been matched with more high ranked players instead, there is a good chance that they would be in the running for first place.

Yeon has only lost one game so far, yet is completely out of the running for first place, since they took their time in early games and continued getting matched with low ranking players.
Season II matchmaking: 3/30/2012 03:12:11


[中国阳朔]Chaos 
Level 49
Report
interesting topic.
I think we should wait for the end result to see the whole picture. Yeon still has 9 games to finish (out of 21 ?), he might be able to end top 3, most opponents are at 2500ish rating.
Maybe an additional pairing system could be used, based on win% or number of games won?
So 2 players who won eg. 9 out of 11 games would get paired, even if their rating gap is big.
Season II matchmaking: 3/30/2012 06:08:37


[中国阳朔] V 
Level 12
Report
@ Ace,
sorry for not continueing this ladder. I'm a lot bussier now, but getting 2 losses at the beginning also made me stop playing. For the reason we're talking about now. I either want to compete for the top 10 or not at all. Maybe I gave up too soon, but that was how I felt.
Season II matchmaking: 3/30/2012 09:46:16

Qi 
Level 55
Report
the only absolute solution: give all members a universal rating based on *all* of the games they have completed. then, only use this universal rating when matching opponents in the seasonal ladder.

i assume such a rating could be created ex post facto. non-members could also be rated, but making it a privilege for members to know/see their ratings on their profile would be neat.

this would serve three main functions:

- the most definitive variable in determining ladder match ups
- the most definitive open seat requirement
- the most definitive indicator of ability (when comparing players)

without a universal rating system, any other means of addressing these issues (ladder match ups, open seat requirements, comparing players) is merely patchwork that falls short one way or another.
Season II matchmaking: 3/30/2012 10:16:21


Ace Windu 
Level 56
Report
How would this system deal with players who have improved greatly over time? *ahem*
Season II matchmaking: 3/30/2012 10:17:38


Ace Windu 
Level 56
Report
V, it's cool I don't mind :)If you don't want to play then I would hope that you don't.
Season II matchmaking: 3/30/2012 12:44:25

The Duke of Ben 
Level 55
Report
Chaos:

Yeon's problem is that they are already scheduled against every player they will be for Season II (not sure how they have 21 games though...). The highest ranked player they will face all season is Nord, ranked 18 with a 2600 rating.

Let that sink in for a moment, and realize that Yeon has zero chance of getting a game against anyone in the top 15. Even if Yeon wins every single remaining game, their wins are not going to give enough rating points to move them up 300+ rating, when half of their games are against people with lower ranking and none of them are against people with really high ranking.

But even if Yeon theoretically could get first place by winning every remaining game, why in the world should they have to? A 20-1 win record should not be required to win against people with 16-4 records. Realistically Yeon will lose 1-3 games of their remaining nine (as just about anyone would, even the highest skilled), and not even get that chance.

Had Yeon blasted through every win as quickly as possible and delayed their loss, they would have been facing top ranked people. Playing games more slowly (and carefully) means not getting pitted against the top players who won their first games and moved on.
Season II matchmaking: 3/30/2012 13:01:11


szeweningen 
Level 60
Report
Well I am not sure that this type of complaining is constructive...

The problems mentioned cannot be realistically solved using current formula of the seasonal ladder without coming back to the problems from last season. Current ladder can be thought of as a variation of swiss system. The main problem is that swiss system is extremely effective, but cannot handle things like joining the ladder late, finding/playing multiple opponents etc. The problems you mention will always remain as long as there can be an imbalance between games played/new opponents found. Fizzer responded to last season's critique in a timely manner and I don't see what else can be done...
Season II matchmaking: 3/30/2012 13:12:46

The Duke of Ben 
Level 55
Report
If nothing else the matchmaking can find opponents further away from your current rank (though not as far as during Season I). I am fairly confident that Fizzer could find an even better solution behind the scenes, but even this fix would be helpful.

I fully realize that no system will be perfect for these purposes.
Season II matchmaking: 3/30/2012 13:26:11

Qi 
Level 55
Report
what might be fun: everyone plays 15 or so games to get a ranking. then the top 64 ranked players are invited to a double- or single-elimination tournament. most people would only play the regular season games. the top 64 would play 1-5 playoff games.

such tournaments are hard to win. but the best tend to rise to the top. and everyone has a chance to win.
Season II matchmaking: 3/30/2012 13:32:41


The Yellow Team
Level 4
Report
I'm not a member and probably never will be so this issue isn't making any difference to my experience/enjoyment of Warlight, but I have been reading this thread with interest.

An idea that popped into my head would be for players to be able to choose their opponents, to a degree. Not 100%, but a system that could run alongside the current one.

Basically, any player could 'challenge' any other to a game that counts towards the seasonal ladder. The player being challenged would have no obligation to accept this challenge if they didn't want to. The current system would still remain, with any 'challenge' game taking the place of one auto game if accepted.

It could reward players brave enough to challenge people around or above them. It could lead to bigger moves in the standings. Players at the top could evaluate the risk/reward of playing someone near the bottom.

I do realise that there would be scope for manipulation of the system... but as that seems to be a hot topic at the minute, it would be nothing new. If people are sneakily taking advantage of flaws in the system now, why not 'legalise' it making the choice of who you play an extra tactical consideration in the ladder?

And then, as I was writing this, another thought popped into my head. Multiple accounts/friends helping each other would absolutely destroy any credibility in the whole thing.

So you should probably just forget everything that I've just written.

Sorry to have wasted your time.
Season II matchmaking: 3/30/2012 15:11:15

Yeon 
Level 61
Report
It's clear that Fizzer has at least greatly changed the formula for matchmaking, as he said he would! Previous season I was at the top through the entire season, and didn't get high enough rated opposition to have a chance at first place (since I won all 15 and came 2nd).

This season I didn't intend to play since I have a lot on my plate these days, but we were all auto-joined and then I decided to play when I had time for games.

I see the point that strategically, I shouldn't have surrendered my lost game until all my games were generated, and I did think about it, but I felt that kind of thing would've been in bad taste (especially so since I played pwn-trockerz who is one of the most prompt players on the site, dragging out his game would be unfair).

Personally, I think there's something wrong with the rating system used overall, but the most constructive suggestion I have is this: When calculating the rating of a player, assume all their opponents have played 20+ games (even if they haven't). Otherwise, one of the biggest factors deciding a player's rating is whether or not their opponents actually finish 20 games, and I don't think that has anything at all to do with a player's skill (in fact, if it does, then the strongest players get weaker rating this way because they crush opponents into losing faith and quit the season).

Anyway, I suppose it's clear that I wouldn't have gotten 1st even with 21-0 this season - and I don't think I will be anywhere close to that.

Good luck to everyone still in the running!
Season II matchmaking: 3/30/2012 16:59:37

Dr. TypeSomething 
Level 3
Report
What about adding a random component to the matching algorithm? Right now it tries to pair you with the person most closely ranked near you. What if there was a random number generated, for example, between -20 and 20 that determined if you would play anybody between 20 above you and 20 below you. Sure some people would get lucky, but over 20 games or so the luck should approximately even out. But basically it would still solve the problem of the very best playing the very worst that happened in Season 1, while adding some chance for mobility. I think there is something like that right now because some of my games have been with players ranked a few spots above me, but the variability could be increased.
Season II matchmaking: 3/30/2012 16:59:49

[16] Jasper 
Level 52
Report
Let me start off by saying that I am not and have not been playing in this ladder, but I was reading I feel I know what some of the problems are here.

First off, the system used to decide ranks is an ELO system. This is a very sound system, however, it isn't exactly used well here.
What the ELO system does is to quantify someone's skill level by giving it a number. Then, what it does is give an approximation of how much chance a player has to win against another player based on the relationship between their ratings and based on that, the points will be given and lost after the game. As such, if your skill level stays the same, so should your rating, and though you lose some games against worse opponents and win some against better opponents, your rating should stay the same.

This system is based on the assumption that the rating of players is accurate. Sure, there may be a slight difference and that will compensated over time by the system, and people's skill levels is supposed to change over time, but if the ratings aren't approximately correct, the system doesn't function.
An example of this would be when in chess a number of players was advancing faster than the ELO system was able to reflect, so they played against opponents that were weaker than them, but because their rating hadn't caught up with their actual skill level the system viewed them as weaker than their opponents. This worked out well for the players who were advancing so fast, as their ratnig went in the right direction. However, their opponents were punished much harder for the loss than they should have been, so their ratings were no longer accurate compared to their skill level. As you may be able to see, the ELO system is a very delicate system that can be tipped out of balance by simple things like that.

Looking at things that tip the balance, we should also look at new players. You will be unable to represent their skill level accurately. They are given a certain entry points in rating. This won't be an accurate representation. However, because relatively few players are new and most players have established ratings that are correct (more or less, anyway) they will settle in at a correct level eventually. Oftentimes the system is also manipulated into recognizing that new player's ratings aren't accurate, by for example, calculating their opponent's losses and gains differently (a different k-factor) for their first games.

With the way the ELO system works in mind, it quickly becomes clear that starting everyone up at the entry score at the same time is going to cause some problems. Yes, you would be getting a decent system eventually, but it would take time before everything is settled down. And that doesn't happen in two months, I am afraid. As such, the rating system doesn't function as it is supposed to when used for the seasonal ladder, which lasts only 2 months. For such short periods, it would be a much better idea to look into rating systems which are meant to be used in single competitions (rather than lifetime-systems, like ELO).

(As an aside, Fizzer also makes a point about inflation on the [wiki page in question](http://wiki.warlight.net/index.php/Ladders). While point inflation is something that happens in ELO rating systems, so is deflation. The thing Fizzer is talking is about something else, though. It is all about the system not having settled in yet. Ratings aren't accurate yet, which means that the entry points given to new players also don't match the average skill level of players. Not yet, anyway. Over time the system will settle and that anchor point that is defined (the points given to new players) will be lined up neatly with whatever the average skill level of new players happens to be. Until that time, there will be inflation to get to that point. Once settled, this inflation will stop and you will have a stable system. However, this does not happen in the warlight rating system, because it has a point drain to no longer taking into account older games. As such, you will have a system that is not very likely to settle at all, and in fact, the change that Fizzer suggests is needed, in fact breaks the ELO system. However, that is all about the other two ladders, as the problem in the seasonal ladder is even more funcamental as I explained before.)

Getting back to the problem here, we aren't done yet. The way the ELO system is used is a big problem, but it is not the only one. The other problem is indeed in what most people are complaining about: the pairing system.

If I understand things correctly, it is based on the swiss pairing system. Again, we are dealing with a very solid system.
The problem, then, is in the fact that it is more than simply a pairing system. It is in a way a rating system. The Swiss system pairs players in such a way that their number of wins is a measure of how well they have been playing at that tournament (or ladder, or whatever). I don't use the word skill here, as that's not what it is about here, skill is a quality of a player, while being ill does not change your overall skill level, it may well mean you play more poorly during a certain tournament (which a skill level should see as a small problem you will be able to get back from in time, while in a tournament you may just finish last). I don't know how well that translates to the system that is used here, but I would imagine something like the number of wins saying something about a player.

The real second problem is that these two system are in one another's hairs. ELO says it doesn't matter if you play against much stronger opponents, as it is already taking that in account. It actually even assumes you play players of varying skill levels to make your rating stay correct. Actually the moment at which it needs this the most is when the system is still settling in. In the mean time, the swiss system makes sure that this playing against player of different skill levels doesn't happen and as such, the settling of the system is actually slowed down. Basically, because they try to do the same thing in different ways, they are ending up getting in one another's ways.
Season II matchmaking: 3/31/2012 04:09:59


[中国阳朔]Chaos 
Level 49
Report
we could have divisions with round robin system where the top x promote to the next division (next season) and the bottom x demote to the lower one. It could take a few seasons before some players reach the top of division 1, but at least they can win on their way up.
Posts 1 - 23 of 23