standard deviation=sqrt(n*p*(1-p))

Darn. :( Thanks.

Alright, so here's the question I began with:

- How do I determine the impact a template has on the likelihood of upsets?

I could simply measure the % of games that ended up in an upset, but a template could end up in 10 games each ending up with a 1490-rated player beating a 1510-rated player and it wouldn't be as worrying as another template that has 10 games with the first 5 involving a 1490-rated player beating a 2000-rated player and the last 5 involving the opposite.

How do I measure the rate of upsets in a way that also accounts for the relative strenghts of the players involved- i.e., weights the upsets based on how unlikely they were to happen?

EDIT: So Corvus and I are having a private convo about this- here's an idea:

Using players' games during a time period to come up with their Elo curve (historical) and use that rating to figure out what sort of bias the template had during a certain game.

So the central challenge here seems to be getting reliable Elo ratings for players to test with.

And actually, I think that tournaments could still be used since we'd be able to come up with players' Elo ratings (reliably) at a given point in time. But that still leaves us with the problem of the template affecting those Elo ratings to begin with.

Edited 6/17/2015 08:07:02

IMO there's nothing more strategic in 16% SR than there is in 0% SR and vice versa
its just that you have to make few adjustments in your attacks(IF NEEDED) to account for luck

with the new updates warlight has made in the analyze attack column its much easier to calculate now and those who were unaware of how it worked earlier can access it with much more ease :)

so basically those who are ignorant have high chances of losing in 16% SR than those(top players) who are not which is an important thing to note because this is somehow misunderstood to be a part of strategy(which i don't like when pro players say it)

^ I agree. The game of Risk is about calculated risk. A small change in the luck factor only adds an equal challenge for both players. The only issue with luck and strategy is how luck modifier changes enable otherwise improbable things to happen (e.g., a player never succeeding at a 3v2 while their opponent always does).

Even hundreds of years later opinions differ still in matter if chess is strategic or psychological game. One should not dvelve into quantitative and qualitative analysis without eliminating first its prejudice.

First of all if you want to measure something (in most cases) make sure you understand what you measure, unless you try to find something new and define it. But in general know what you measure and have a definition beforehand to make it stick, to eliminate options for future interpretation and alteration of results.

Skill, luck, strategy? Are they opposite, co-exist, consist or what?
Good or bad? Unless you do not give any value or clear definition its merely your personal opinion and only dilutes your study/research/method etc.

WR/SR, and luck factor, blind factor?

There is probably no universal definition of strategy, but it does not need to mean that we cant measure it or define it somehow. In general and to simplify I ususally break it up for two: Pure-static strategy and Comparative-Dynamic strategy.
Pure-static Strategy - Chess like gameplay - where all information is known and visible, no random factor unless playing black or white; falls under formal logic, which allows to use in-depth move analysis due to open intel factor- every move has clear and visible effect on further game and can be put into test and under judgement. To sum it down: all possible alternative opponents choises are known to you and their current status of things.
Comparative-dynamic Strategy - More of a general skill how one operates under certain rules. In sense of this partiucal game I would give definition close to that one has to take into considerations all rules, variable factors and changing evironment in closed system. (Closed system: meaning defined map - knowing its not endless; variables: starting positions eiter manual choice or random, fog - which consist blind factor in genral, and rule based variables: luck and WR). To sum it down: Strategy lies in estimating opponents position (calculation risk) under blind factor rules in closed system. [Basic topic-discussion lies what are the factors that give 100% winning advantage to player and if it is true in terms of independent of opponents player choice to avert-prevent it; what measure of rule, luck, random is avert of strategy - if any?]

To give example. Straight round would be definitely Pure Strategy. Weighted random more of second kind as its rule varies and applies differently to every attack and player. Same goes to Fog, automatic starting positions, random wastelands and luck (although here lies the discussion about how big luck and where to draw the line between lottery - hope general logic solves it for now). Of course if we reduce strategy merely on choice, then one of course can claim that buying two lottery tickets will maximise your chance of winning - which in essence is just absurd to define as strategy.

I do not offer full definition or guideline and its merely my opinion although put togethet from various sources and models used in different platforms. Tried to add small WarLight background into it. Basic thing to keep in mind: if you start measuring something just define it, know what you measure and make it logic. Do not leave room for interpretation or mis-results. Mainly: What is strategy to you, or what kind strategy you measure in particular?

Logic is great tool once perfected. In strategy games and analysis one should know how to use, understand and define: system (open-closed), game rules, variables, players-choices, independent-dependent factors. How they supplement each other or actualise in real terms.

Edited 6/17/2015 10:14:26

Returning back to what Corvus said:

1. I've come to agree that the central issue is the unreliability of the Elo ratings. We can't truly know what an "upset" is (or how big an upset is) unless we have reliable Elo ratings for each player. He does have a great idea for getting good baseline Elo ratings, but we'll need some players who're interested in helping us out there.

2. That said, I think the bias ratings are still a lot more than just rounding error. Even when Elo ratings are tainted by a template, the rate of upsets is going to differ significantly. For example, a 2000-Elo rated player (on a lottery template, someone who just got lucky a bunch of times) is still going to lose 50% of games against a 1500-Elo rated player on the same template, but a 2000-Elo rated player on a purely strategic template is going to win a far greater share of games against a 1500-Elo rated player on that template- so even with tainted ratings, we can still meaningfully measure the difference in the probability and magnitude of upsets in a way that's not just accounted for by the Elo ratings. Bad templates still have a greater (or lower) occurrence of upsets, even when just measured in terms of their tainted Elo ratings- simply because of how these templates work.

In other words: what we're actually quantifying is the accuracy of Elo ratings produced by a template. These Elo ratings are predictive- they make claims about what portion of head-to-head matchups two players are going to win in the long run. If the template is bad, it's going to lead to bad Elo ratings- ones that don't actually reflect how likely a player is to win against another player with a certain rating. And we can test that- by looking at the data I'm analyzing and figuring out the rate and magnitude of upsets.

It's like testing a hypothesis- how accurate are the predictions? The hypothesis in this case, of course, is that a template is purely strategic. And the prediction that hypothesis makes is that the outcomes on that template are going to closely reflect Elo-based predictions.

Good template -> reliable Elo ratings -> Elo ratings that are more consistent with actual overdog win rate -> lower bias rating

Bad template -> unreliable Elo ratings -> Elo ratings that are less consistent with actual overdog win rate -> higher bias rating

As far as that "tainting" issue goes, I think it's actually still sound. Perhaps an analysis of a single-template ladder or tournament would actually be more sound based on this logic than an analysis of a single template on a multi-template ladder or tournament, because the Elo ratings in that case would better reflect the impact of the template.

That said, the long-run analysis issue still persists- when we have a consistent Elo rating for each player rather than an Elo curve (which we actually do have in CSL, but right now that's got its own issues), we won't be able to accurately deal with games where the overdog lost because they were just not as good back then as they are now- for example, my first many games on the ladder would probably be considered upsets now since my rating has gone up significantly from then. But in reality, they weren't upsets- I was just a shit player (still am, but used to be even worse back then). There's two solutions to this- either we analyze fewer games or we assign new Elo ratings using a separate process that reflects a player's development. The second is more reliable.

Finally, I think Corvus's process is going to be the most sound solution- we pick 20 or so players, analyze them to get baseline Elo ratings after some realtime games played in certain conditions, and then we use them to test templates. Those baseline Elo ratings would probably be the best way to measure upsets, but there's still the assumption that players have a single skill level across all templates instead of varying skill levels unique to each template. If you find that assumption too big to make, then my original analysis would be the most valid (since all Elo ratings are based on a single template).

I was expecting someone on my blacklist to pop in and tell me to do something better with my time. >_<

If you're hanging out with the right crowd, you won't get that kind of troll.


JSA's the closest thing Warlight has to a historian. If the two of you team up, then maybe you actually will disprove gravity

to answer your other questions knyte :-

is EU 4x5 0% WR just as good as EU 4x4 0% SR?
No!EU 4x5 0% WR simply sucks

Is Rise of Rome too big to be a good 1v1 map?Is it a good 2v2 map then?
it depends from person to person whether they like it or not.i personally like RoR 3v3,4v4 and 5v5
i hate the idea of 1v1 but if its a coin game,any time dude :)

Are Poon Squad's settings really that bad?
consider my 1st answer.now multiply it by 1000

How do I explain to someone that Guiroma 1v1 is actually not a "really weird and bad template" but in fact a solid and well-tested strategic 1v1 template?
don't.if they don't realize it after playing 4/5 games then don't even bother

Edited 6/17/2015 14:08:53

can I get a link to the guiroma template?

Guiroma 1v1 https://www.warlight.net/MultiPlayer?TemplateID=668568

Guiroma 2v2 https://www.warlight.net/MultiPlayer?TemplateID=671768

Very interesting thread! And thank you Mr Tranquilizer for an especially interesting post. I am now going to throw some further fuel into the fire. Before that let me say that the basic premise of the thread, that a good strategic game should give the better player a proportionately better chance of winning, is true.

Chess is in the end not a game of skill (nor of strategy). We have computers which have inherently no skill and no intelligence now beating world champions. So chess is a game of dedication of computing time. If a human player could spend enough time to analyze all possibilities, they would win over one who was not willing to spend that time, regardless of skill level or intelligence. The same goes for 0%SR games, they ultimately become measures of dedication rather than skill or strategy.

In human terms, we do not have time to analyze all possibilities, so at the end of our analysis range, we have to apply a judgement of how likely the resulting position is to give good results. That is skill and strategy. Lightning chess is a good measure of skill and strategy. Another example is backgammon, where you cannot know what moves you will be able to make. You have to place your pieces so as to get the best chance of getting good subsequent outcomes. Skill and strategy. But also possibility of upsets.

Another factor is whether a game is interesting and enjoyable. Basically, if you want worse players to continue playing, they must have a chance to win and at least feel good for a large part of the game. Even chimpanzees can work this out, if nobody plays you can never win.

Yet another factor to take into account is that "100% luck" in warlight does not make it a lottery game. You still have the basic attack/defence probabilities and a bigger force will have a bigger chance to beat a smaller one. So the skill is making your bigger force appear at the best place. Perfect example of strategy.

So I propose the following with regard to strategicality of settings:
- luck percentage doesn't matter, it just changes what you have to account for, and 0% is more likely to reward time spent than actual skill.
- Same for SR versus WR
- A small upset should not leave you screwed. E.g. opponent is lucky and gains a bonus first move and you do not. In a good strategic game, you should be able to recover from this and your greater skill be able to compensate for a bit of bad luck.

One final unrelated thought on team games: a team is not just the sum of the skills of the players, there are other things that come into play.

Like teamwork.

"""
IMO there's nothing more strategic in 16% SR than there is in 0% SR and vice versa
its just that you have to make few adjustments in your attacks(IF NEEDED) to account for luck

with the new updates warlight has made in the analyze attack column its much easier to calculate now and those who were unaware of how it worked earlier can access it with much more ease :)
"""


I(and others) meant 0%WR over 0%SR. 16% SR over 0%SR isn't that much of a difference other than a few differences at attacks with higher armies as you've stated.




"""so basically those who are ignorant have high chances of losing in 16% SR than those(top players) who are not which is an important thing to note because this is somehow misunderstood to be a part of strategy(which i don't like when pro players say it)"""

I fear you make an assumption here that you have enough income to always play the perfect move. The "strategy" comes into play when you do not have sufficient income and must decide how to balance risk vs reward.
i.e, not every move is worth attempting with a 4v2, but some absolutely have to be. You must decide carefully, how you attempt expansion/exploration etc.


P.S - This is actually a good research area. If you have free time, check this article (https://en.wikipedia.org/wiki/Multi-armed_bandit)

@ Corvus5

"""
there is ways to measure that error e.g. "Binomial test"
lets make a simple calculation based on your numbers
the expected value for tails in your example is 5000 and the mean average error is 50
That means inside the intervall [5050,4950] -times Tails we have an accumulated prabability of 68.3%
and in intervall [5100,4900] -times Tails we have an accumulated prabability of 95.4%
and in intervall [5150,4850] -times Tails we have an accumulated prabability of 99.7%
so if you get outsides these intervalls (especially the last one) the probabillty that your coin was biased gets bigger all the time since its very improbable that you got soch a big deviation from the expected value
"""


I see. But in this case,what is my expected value for a player(is it the Elo rating?.. and if so how do I use the Elo rating to measure the error, as well as use to it determine the deviation from the expected value(so as to determine the "strategicness")).

I'm a bit confused now, as I feel we're starting out with a statement and using it to prove that it is correct.

@MOTD:

Your expected value here would be the overdog win % derived from the Elo difference. Your actual value would be the actual % of the time that the overdog wins. So you can figure out how far the actual value is from the expected value in terms of standard deviation/standard error in order to figure out how likely it is that you would've gotten this actual value had the expected value been accurate for the template. So if the actual win % is a certain distance away from the expected value and the probability of your getting that win % is, say, <5%, then you can be reasonably certain that your expected values don't describe the template's actual results- i.e., the template is "unstrategic."

What Corvus pointed out is that it's rather odd to use the Elo scores to derive expected values since they themselves would be affected by a flawed template.

Which made me realize that I flipped the test- the expected value should instead be the actual win rate. The measured value is the Elo score- what we need to be measuring is how accurate the Elo score resulting from a flawed template is going to be in terms of matching up with the expectations set by the actual win rate. (See my post before this).


However, where I'm stumped here is how to get the standard deviation- which sample are we calculating the stdev./stderr. for?

So how do I go about quantifying the improbability of results?

I mean, I can't just straight up multiply the odds of getting the results I did end up getting- that would lead to some odd results (try multiplying (.5)^2000 to represent the probability of getting any certain set of flips from a coin). And I can't just use combinatorics to sort that out since there's too many players for me to just reorder things and sort them out that way.

All I really want to calculate is P(I got the results I got|the theoretical model I'm using here is valid) to get something really useful. But I have no idea how to do that.

Edited 6/19/2015 05:33:17

New idea: Literally just calculate the probability of getting the results you got.

This needs to be modified to calculate cumulative probabilities of getting the # of overdog wins you got or more overdog wins- ie, to measure how far you're deviating from the center

So we have the probability that the average overdog beats the average underdog.

Since Elo probabilites are based on the difference in rating, we also have the average difference in rating (or, in this case, we take the difference in the average rating- which should be the same as the # of overdogs is the same as the # of underdogs, so the average rating difference is the same as the difference in average rating). We end up using this to calculate the average probability that an overdog beats an underdog (which, in my understanding, is the exact same as the probability that the average overdog beats the average underdog- once again, because in this case the average rating difference is the same as the difference in average rating).

Now, once you know P(overdog win) and P(underdog win), you can start treating it like heads and tails and use combinatorics. So you end up using the formula:

P(my results|the Elo predictions made based on the template are accurate) = 
P(overdog win) ** (# of overdog wins) * P(underdog win) ** (# of underdog wins) 
* C(# of games, # of overdog wins)

This gives you the probability that you would've gotten the exact # of overdog and underdog wins that you did end up getting, based on the template.

Main assumptions here:

A strategic template would yield reliable Elo ratings.

Over a set of games, you can accurately model the probability that an overdog wins each game and the probability that an underdog wins each game based on the average probability of overdog/underdog wins across all the games. In other words, if you know that the overdog, on average, wins 60% of the time across all the games, then you can accurately model the games by saying that each time the overdog has a 60% chance of winning. I don't think this is a farfetched assumption- if, over 1000 biased coin flips, you have a 90% chance of flipping heads on average, you can treat that as saying that, for each coin flip, you have a 90% chance of flipping heads.

So if your Elo ratings are bad, your resulting P-value should be low (e.g., your Elo ratings predict that the overdog will win 90% of the time but in reality the overdog only wins 60% of the time, so your template is bad and has led to inaccurate Elo ratings and a low P-value)

Edited 6/19/2015 06:49:29

Pure skill setting is enough to obtain an equal game.

Derp. Figured it out and ended up doing just what Corvus hinted at (calculate the probability of getting results that are x # of standard deviations from the mean). I'll apply this to ladder data at some point, I promise.

It's implemented in CSL, at least, for now.

Edited 6/19/2015 10:18:08

Pure skill setting is enough to obtain an equal game.

It may be an equal game in some perspective, but equal does in no way mean strategic. A lottery game, albeit pure skill settings, is still not a strategic template. The question here is wether it is possible to measure how strategic a certain template is.