http://www.youtube.com/watch?v=XCpIR7T1VgA
This is some footage from pax about CSR, and one of the things they talk about is level locking. Level locking is not real, It has never been real, and isn’t possible. So please do not worry about level locking, It does not, nor has it ever existed.
Level locking is an inherent problem in any TrueSkill based ranking system due to the mathematical variables that control the system. Basically your skill is evaluated as a function of your performance compared to the system’s expectations for you (called delta in the XBL algorithm I believe) and the system’s “certainty” with which it places your ranking (called sigma in the XBL algorithm). Essentially level locking is when the system becomes so certain of your rating (based on evaluating hundreds to thousands of your games) that it can accurately predict your performance within an incredibly small range; eventually that range falls below the threshold which could allow you to increment ranks up or down. At that point your sigma can become so difficult to overcome that you could need streaks of 17 wins in a row to go up a single rank. And this isn’t even the true definition of level lock because that number eventually becomes so large that it is impossible for you to actually play enough games to overcome your sigma which therefore puts an upper cap on your skill. Basically, I just wanted to say that level locking is very real and, as Bravo said, the best way to avoid the problem is to have scheduled resets so that your sigma can reset along with your rank and you could conceivably “break through” your cap.
tl;dr Level locking is real because math.
> tl;dr Level locking is real because math.
I’ve done the math, and have evaluated the trueskill system, nor would have I of posted this if I didn’t know otherwise.
Show me an account that you think is level locked, and I’ll show you how it isn;t.
I played > 1,500 Lone Wolves matches in H3 on this account. I can confirm that ‘Level Locking’ is a myth but it takes a LOT of consistency good or bad to affect a change to whatever skill level it is at.
You can win 5 games in a row and think you are going to increase in skill but then suffer 1 poor game and you need to start again. There’s a lot of math about expected outcomes from opponents that comes into the equation also. Most people don’t have the time to understand the complexity so just say - I’m rank locked - when in reality they just aren’t as consistent in winning as they think they are.
I did play Team Slayer against some people in H3 who always went in as a T04. True Skill had troubles with them, after 320 wins out of 325 games they were only mid 30s in rank. They were awesome.
Had to be a moron to even think level locking was a thing. I got many people out of their so called “level lock”. You just need t keep winning
For those of us who are still a bit new to the whole debate, would any of you mind explaining what “level locking” is? Given Evil Snoopie’s post above, it seems like an interesting subject, and an explanation could help to get others interested in the topic.
Microsoft uses applied statistical theory in its TrueSkill system to match and rate players.
There’s a lot of math behind the system but what it boils down to is the statistical significance of an event. The system is trying to work out on average how good you are against a pool of opponents and give a ranking based on that. All that matters is win or loss and relative ranking against the player pool you face. Kill Death ratio doesn’t matter, just your win record. Halo Reach tried some freaky things with win/loss, kills, deaths, assists and other matters that was hugely unpopular and ‘gamed’ by some people.
Eg I am rank 35. I play FFA against people ranked below me - 28, 32, 30, 27. I am expected to beat them based on my rank so if I win it was an expected outcome so it isn’t considered statistically significant but adds weight to me maintaining or increasing my rank. Alternatively if I lose it is statistically significant as that was not the expected outcome and I may decrease in rank. As you can see given the relative ‘skill’ of players there can be a myriad of potential outcomes given relative positioning at the end of the match.
The concept of ‘Rank Lock’ becomes important when you have a significant history within a playlist. Take my example of over 1,500 Lone Wolves matches. I reached a peak of 48 in Halo 3 and in order to achieve that I had to earn about 10 matches of wins in a row to move from 47 to 48. My history had created such a huge amount of inertia that it took a significant streak to overcome the average of my performances to get there. I was never locked, it just took a lot of consistency to overcome. The converse applies, once you are a higher rank loses can have a big impact against lower players.
This lead to why so many people created 2nd accounts in Halo 3. You start anew and then ‘appear’ to rush through the ranks like you never had before. Your Trueskill is significantly higher than where you start from and you are unburdened from the weight of history of your previous account. There is no average performance so statistical uncertainty is high and the effect of winning is greater - people rank faster than an account with hundreds of matches.
An economy developed in buying and selling these ‘new’ highly skilled accounts. Doubles 50s in Halo 3 could be achieved in under 25 matches given the right conditions. Frankie has stated this is one of the key reasons he wants to avoid a visible CSR and why Halo Reach had Seasons for its ranking system instead of permanent playlist rankings. I’m not making a judgement of these just pointing out some facts.
To a degree putting every playlist with CSR is somewhat of a concern. Every match is important, against Halo 3 where ranked and social existed where one had an impact and another didn’t. JIP is going to be hugely important here also. No-one wants a loss joining a losing team or joining an FFA match with 5 kills to go. I don’t understand how the win/loss system is going to take this into account.
There’s heaps of other issues also and anecdotes that I’m sure people have. TrueSkill struggled with teams sometimes - I saw a team of 4 that played exclusively together who appeared ‘locked’ at mid 30s in Team Slayer for Halo 3 even though they had a 99% win rate after 350 matches (I was convinced they were 50s just gaming the system). The use of negative EXP or low skilled accounts throws the system out.
In short - what is the statistical significance of an outcome with reference to previous performances?
> For those of us who are still a bit new to the whole debate, would any of you mind explaining what “level locking” is? Given Evil Snoopie’s post above, it seems like an interesting subject, and an explanation could help to get others interested in the topic.
The short and simple version: It’s when you can no longer increase your rank beyond it’s current level (or at least believe that you can’t).
The slightly longer version:
TrueSkill uses 2 variables, they are known as mu and sigma.
mu: is the numerical representation of your current skill level (your rank)
sigma: is a representation of what your skill could be. It’s also referred to as ones ‘uncertainty factor’. As i understand it, your sigma could be described as how much higher (or lower) the system believes your actual skill could be and acts as a multiplier when your TrueSkill is recalculated after a game (how much your TrueSkill changes depends on your mu, your sigma, the mu of your opponents and if you win or lose). Because your sigma it’s a multiplier it means that when your sigma is high you will rank up (or rank down) faster than when your sigma is low.
When you first start playing your sigma is high because the the system has no idea of what your skill could possibly be. As you play more and more games the system is able to build a profile of you. The more consistently you play the lower your sigma will be. For example if you win 50% of the time and lose the other 50% against people of the exact same skill then then the system will believe that your mu is correct and therefor your ‘uncertainty factor’ is no longer uncertain.
Because of this, people who have played a lot of games find it hard to continue to rank up simply because there sigma is low in fact it can become so hard to rank up that it almost seems as if your rank is locked.
The only solution to this scenario is to increase your sigma which can be done in 2 ways (more or less).
(A) Win a whole lot of games against people of much higher skill (Which is more than likely impossible as most people in this situation are probably between 40 and 49 and there is no such rank that exists which is ‘much’ higher. 40-50 is close enough for the system to match against each other so I assume that it’s not enough of a gap to cause a significant increase in ones sigma)
(B) Lose a whole lot of games against people of lower skill. (Which is something many people are just unwilling to do)
I’ve never heard of level locking until PAX.
Could level locking be avoided if the system only looked at your most recent, say, 20 games, rather than hundreds or thousands? That is, determined your current CSR, moving it up or down, on a smaller, regularly updated, sample size. Doing so would surely avoid the so-called level locking.
> I’ve never heard of level locking until PAX.
>
> Could level locking be avoided if the system only looked at your most recent, say, 20 games, rather than hundreds or thousands?
Last 50 or 100 might be more appropriate but your concept is correct.
thanks for the breakdown folks
the last 20 might just be a few days…
Make it 500? That’s just the month or so…
> eventually that range falls below the threshold which could allow you to increment ranks up or down. At that point your sigma can become so difficult to overcome that you could need streaks of 17 wins in a row to go up a single rank. And this isn’t even the true definition of level lock because that number eventually becomes so large that it is impossible for you to actually play enough games to overcome your sigma which therefore puts an upper cap on your skill.
This post has been edited by a moderator. Please do not post comments that are discriminatory in nature.
*Original post. Click at your own discretion.
This is untrue. Trueskill has a constant called tau, which prevents your sigma from falling below a certain threshold. Every player will retain a certain amount of fluidity in their rank no matter how many games they play.
And in Halo 3, there were some cases where a Team of 4 very good players would jump into Team Slayer on fresh accounts, win 100 games in a row, and get stuck at level 14. However, this was caused by Bungie’s limited matchmaking implementation, not Trueskill’s core itself.
Essentially, bungie’s system would go “4 dudes with mu [40], sigma [0.7], okay, Go play these mu [14] sigma [0.1]s”. After the blowout win, Trueskill would say “umm the skill(mu) levels are so different this result gives me no information, I’m not changing these guys’ ranks”. Repeat forever.
> Could level locking be avoided if the system only looked at your most recent, say, 20 games, rather than hundreds or thousands? That is, determined your current CSR, moving it up or down, on a smaller, regularly updated, sample size. Doing so would surely avoid the so-called level locking.
Basically, it already does that.
Your history is already compressed into just two numbers: the mu (mean skill) and sigma (uncertainty). It is updated in successive approximations after each game. So your recent results are ALWAYS weighted more heavily than previous results.
In addition, a small amount is added to your sigma (uncertainty) before updating your skill after each game
The apparent level locking in Halo 3 was a combination of several factors:
There is some other stuff, but this is kinda hard to explain quickly, and you guys haven’t even bothered to do even the vaguest amount of research. Seriously, just but ‘TrueSkill’ into Google and read what comes up. Half the stuff people get wrong when talking about it is already clarified in the description or FAQ.
> And in Halo 3, there were some cases where a Team of 4 very good players would jump into Team Slayer on fresh accounts, win 100 games in a row, and get stuck at level 14. However, this was caused by Bungie’s limited matchmaking implementation, not Trueskill’s core itself.
>
> Essentially, bungie’s system would go “4 dudes with mu [40], sigma [0.7], okay, Go play these mu [14] sigma [0.1]s”. After the blowout win, Trueskill would say “umm the skill(mu) levels are so different this result gives me no information, I’m not changing these guys’ ranks”. Repeat forever.
Are you talking about Big Party Matching?
level lock -
synonymous: true skill has found your skill level accurately.
used: as an argument to prove that true skill doesn’t work.
> level lock -
> synonymous: true skill has found your skill level accurately.
> used: as an argument to prove that true skill doesn’t work.
This!
I don’t know how many people said they were level locked, and after taking over their account, only took 1 or 2 games to level up and keep leveling.
In H3, if you stayed around the same K/D and win loss ratio you didn’t move. Hence, YOU ARE WERE YOU SHOULD BE!
When I got the account the K/D went way up and the win ratio also raised quickly. So the system moves me up. I remember an account that he said "I’ve been stuck for 2 months on 41 and 42. I got him to a 50 in about 20 games and didn’t have a boosting in the room.
> > And in Halo 3, there were some cases where a Team of 4 very good players would jump into Team Slayer on fresh accounts, win 100 games in a row, and get stuck at level 14. However, this was caused by Bungie’s limited matchmaking implementation, not Trueskill’s core itself.
> >
> > Essentially, bungie’s system would go “4 dudes with mu [40], sigma [0.7], okay, Go play these mu [14] sigma [0.1]s”. After the blowout win, Trueskill would say “umm the skill(mu) levels are so different this result gives me no information, I’m not changing these guys’ ranks”. Repeat forever.
>
> Are you talking about Big Party Matching?
No, but ‘Matching using Conservative Skill Estimate’.
If I can take a so called ‘level locked’ colonel account in halo 3 and get it to 50 there is no such excuse for why a player could not get to 50.
You just have to win more and be better. If a player is not good they deserve to live in their fanatical mythical world of level locking to shield themselves from the truth that they just can’t get to the top due to lack of skill.
> there is no such excuse for why a player could not get to 50.
Maybe they don’t have the skill level to get there.
MM isn’t intended for everyone to get to 50. It is there to match players of similar skill and provide fair and even competition.
> > there is no such excuse for why a player could not get to 50.
>
> Maybe they don’t have the skill level to get there.
>
> MM isn’t intended for everyone to get to 50. It is there to match players of similar skill and provide fair and even competition.
way to take part of my sentence out. I said if someone says an account is level locked it really isn’t. Because someone out there could take the same account and get it to 50. A player may feel level locked but it’s their lack of skill that is preventing them from ranking up. They think they should be a higher level when in reality they are where they belong. If they were better they would win more matches more often ranking them up.
and you are right, not every player is meant to get to 50. But a lot of players have ruined 1-50 because they cried about not being able to get to 50 hence the reason we no longer have visible trueksill in halo games which is sad.