The whole mmr thing makes no sense, because your actual rank means nothing. A diamond 6 should never go against an onyx 1700, they should be facing other diamond 6’s. Most of the games I have played i am facing 4 onyx players while i have one other onyx on my team and 2 low diamonds.
The game expects you to carry because you have high mmr, and it makes the matches extremely inconsistent. They should get rid of the mmr and match you against players who are your rank. If not, there is no point of even seeing what rank you are, it’s a useless number.
That’s one of the most important issues. This is the first Halo where a highly skilled player can’t carry. It used to always be a close 50/50 between individual skill and teamwork. Not anymore.
Your MMR and CSR are usually pretty close. The only time they really drift apart is with placement.
So, your rank still means a lot.
Not a fault of the ranking system.
More a problem with the match-making / low population.
Diamond 6 vs what is essentially Onyx 5 is a big gap. But, on the same token it’s potentially the same as a Diamond 1 vs Diamond 6 (which doesn’t sound as bad).
And often, wide skill ranges in the game are the fault of a group of players who have squadded up. It sucks, and I wish they would put a range limit on squads.
Unless you are in the post-placement phase your CSR and MMR will be very close.
But regardless. Your skill level is your MMR. So you are only being asked to carry vs your expected skill level.
If the system is currently over rating your MMR then don’t worry. You will lose a couple of games vs tough opponents and your MMR will fall. It won’t take long and it should settle into your skill-ceiling.
Instead, take it as an opportunity to win a few games up and rank up that CSR.
Dude this is just false. I’ve been matched against a 2200 player and he had a lietral bronze on his team. We were all dia/onyx, he had a 1500 a plat and a bronze. The game weighted the mmr values and said this averages out to within parameters, throw it together.
The poor guy just couldn’t kill us faster than his teammates fed.
Here is the problem:
If you hard carry, the system looks at that and says great job, you’re better than expected, mmr up. Next game it needs to find you a match with balanced mmr, so it is going to find you better enemies and/or worse teammates. Carry even harder. Repeat until you literally can’t carry the game to a win.
But wait! While you may not be winning games, your performance is still great. And mmr doesn’t care if you lost the game, it’s trying to rate your personal skill. So your mmr isn’t going to go down. If anything it’s going to go even higher as you are increasingly soloing the enemy team. You’re just going to keep being forced to carry until that performance starts to suffer and bring your mmr down.
Guess what happens as you approach the top of the skill curve? Guess what happens to the people actually at the top? Not a lot of fun.
Out on a limb here, I’d guess that a similar thing is happening at the bottom end of the curve; as you do worse, your mmr drops, and you are progressively more likely to get better teammates and worse enemies, reaching a point where you have to actively throw if you want to lose games. So you stop losing. But you aren’t actually performing better, you’re being carried, so your mmr will stay the same. So you keep getting put into games that are basically a free win, your winrate goes up, and you start getting annoyed that you win most of your games but never gain rank.
Either the top or the bottom, this kind of thing is happening because the system is only concerned with finding a perfectly balanced match. It doesn’t actually care what your win rate is, it isn’t trying to force players to 50% win/loss, all it wants is a match that it thinks could go either way.
The negative effects snowball; the top players are miserable because their games are an unforgiving treadmill, the bottom players get mad because they win games and don’t gain rank, the people in the middle are at the whims of fate (and a less amplified version of the same problems, depending where in the curve they fall) getting to either carry an easy lobby or be the deadweight in a hard lobby.
That’s how you build a system that everyone hates. Easy fix, just do what pretty much every other game with a ranked mode does and match based on rank. That is, after all, the point of having a rank.
Your MMR is a curve (not an actual number). Wide and flat when the system lacks confidence and tall and narrow when it feels it knows your rank.
Your CSR is matched to the left hand side of the curve. Minus three standard deviations. While your skill level is probably closer to the mean (middle of the curve).
So, when the curve is wide they don’t match - but the more consistently you play the faster the curve narrows and minus three standard deviations gets closer to your probably rank.
If you want citations, it’s nothing particular; the TrueSkill2 discussion paper. the GDC presentations (I think they were 2016). wikipedia about ELO etc.
I’m not sure why people get hung up about the MMR being hidden? There is no conspiracy theory here. It’s a curve, with a completely arbitrary x-axis. You need a way of describing it - hence the CSR. It doesn’t matter 1-1800+, Bronze to Onyx, 1-50.
I guess it’s a bit confusing how they base the CSR on the left hand side of the curve and not smack bang in the middle. But it’s to protect the player. It would be much worse to come out of placement (where your curve can still be quite wide) and have a rank that’s too high. Losing it is far more frustrating then never getting there.
What you are describing is not the fault of the ranking system.
It’s the match-making. And it probably had no choice as I would imagine that the Bronze player was squadded up with the other players.
Some even use it to try and milk the system. In my placement games I came across a team (five times!) who had two Onxy players, a Gold smurf, and a Silver “sandbag”. The latter was purely there to drag the peak player on my team down to high Platinum. The Onyx player’s had a feast.
What you are describing here is literally the job of the ranking system. Find even games so that everyone has to “carry”. If you shine then find harder opponents and give you a chance to prove you have improved.
Rinse and repeat.
When you get to the stage that you are carrying as hard as you can and are sitting around 50% W//L and 1.0 K/D then the system has done it’s job.
I’m really not sure what you mean by “worse” team-mates. The system isn’t going to deliberately mis-match you. It’s probably the great intangible in any match-making system - predicting how random players of similar skill are going to function as a unit.
But what you are probably finding is, as you ascend the ranks, you are coming up against more and more squads who are actually communicating and functioning as a team. In comparison, your random buddies, just can’t compete on those levels.
There are certainly weightings for personal performance. But the emphasis is still on the win. If you keep losing your MMR will go down. And keeping in mind that a mature account has a narrow curve that tends to resist change in either direction.
But again. Just as you describe. If your performance starts to suffer (for whatever reason) your MMR will start to fall. That’s what a ranking system should do.
At the top of “THE” skill curve… or at the top of “their” skill curve.
It won’t be pretty in terms of W/L or K/D which should trend to 50% and 1.0 respectively. But good close games where you have to work your -yoink- off should be the definition of fun here.
It’s the same system across the board.
Of course your MMR should drop as you do worse.
I’m still not sure why you are focussing on “better team-mates”? The system isn’t doing anything deliberate here. But yes, worse enemies.
And yes. When you do reach your rank… you will have to actively throw to lose. Just like you will actively have to carry to rank up.
That’s how the system should work!
I think what you’ve essentially described is forming a smurf account.
Yes, you get a couple of free wins. But the system will pick you out pretty quickly (eg. the weighting on kill rates).
I guess you could just eke out a life in limbo - just doing enough to win and staying at that rank. But why?
Again. Isn’t this the overall role of any ranking / match-making system?
And they actually build in a bit of lee-way either side - so that the matches aren’t all 50:50 - otherwise it can be very hard to rank up / down.
Why? Because they have to play other top players?
They may get mad because they think they should be a higher rank then they are. But if they want to rank up then they need to win the games vs higher ranked teams.
I’m pretty much in the middle… and most of my lobbies are pretty evenly balanced. We occasionally get an Onyx player (usually attached to their sandbagging buddy). And I don’t really recall a lot of easy carry.
Match-making is based on rank. Your rank is your MMR. And we know that TrueSkill2 is pretty good at this.
The CSR is just a representation of your MMR.
I think it would be cool (and educational) if they showed your MMR curve. And they could put an arrow for where your CSR is sitting in relation to it.
It’s hard to look at the CSR on the table and make a judgement on the teams though. A lot of players are peri-placement (MMR could be higher than CSR) and there are weightings for squads and form. It’s not as simple as adding up the average values you see on the screen.
Just a quick look at HaloTracker and you pretty much have a W/L of 50% and a K/D of 1.0. Sounds like the system is working perfectly for you?
It probably should be K/min as opposed to K/D per se.
But that is only if you keep winning. A good performance is only going to minimise the damage of the loss. If your team-mates aren’t consistently picking up the slack and getting the wins your rank is heading South.
I read the trueskill paper (https://www.microsoft.com/en-us/research/uploads/prod/2018/03/trueskill2.pdf), and nowhere do they indicate that your skill as determined by the system is a Gaussian. Rather, they clearly state:
They’re assigning the player a number, the mean of the Gaussian approximation of the joint distribution of match results to marginal distribution of the skills of all players with respect to time, as a skill rating. The specifics of the skill variables, time respective function, results metrics, and how much data they are computing each time they do this(i.e. are they taking data from all previous matches from all players or are they sampling estimated skill over some granulation of time from that single match for the population of the match) are rather opaque in this paper, but I don’t see a need to go dig into the specifics of their expectation propogation algorithms.
They use a curve to compute your rating, but your rating is a number.
And I really don’t see what the Elo system has to do with this, since they’re not using it.
I think I’ve seen you say something like this somewhere before. I didn’t bring it up then, but I have to know:
Why are you talking about 3 sigma? Do you even know what that means? And why would they put your csr 3 sigma lower than the mean? Do you have a source for that?
Because the normal range on a Gaussian is 2 sigma from the mean, where 95% of all data observed are expected to fall. 3 sigma is the range within which 99.7% of data are expected to fall. Even assuming I’m completely wrong in my reading of their paper, why on earth would they put your rank a flat 3 sigma behind the mean of your skill, at the very bottom of the confidence interval???
If everyone’s carrying, no one is. Carrying is what happens when a subset of players on any given team are playing markedly better than the other members of the team. That player(s) carries the rest of the team on their back, making up for their poor performance and winning the game for them, or trying to.
And trying to make people carry is terrible matchmaking design. If you are performing better, It should be trying to find you only matches in which everyone is performing around that higher standard. Secondarily, you want to average out those players, potentially making the best players “carry” as a side effect, to ensure the match is minimally uneven, but the primary role should never be to punish players for doing well.
A system working correctly will try to get you to a moving average of 50% win/loss… It will do this by placing you with better players both on your team and against you, and if you fail to keep up that’s on you, not by giving you a lopsided set of teammates and enemies to try to force every individual game to be perfectly balanced.
Because even if they thought they could perfectly compute everyone’s instantaneous present skill by extrapolating from past data, a Sisyphean task if ever there was one, a competent designer would realize that they shouldn’t try. Doing so will only result in what you describe; high performing players giving it their all and trying to climb will be forced to struggle through (in theory) a perfect 50/50 winrate created by either giving you, on average, teammates whose average skill is less than the enemy team’s average skill.
See above.
Citation needed.
The skill distribution of the population of people playing the game.
Definitely closer to those numbers than they should be. And good, fun games are made by creating a rough balance that can be overcome by individual skill or teamwork, not by forcing one player to 1v4 if they want the win.
Nothing deliberate, just the negative effects of bad design philosophy. I focus on better teammates because, as above, that’s the problem. Forced 50/50 is like that.
Nooooooooo dude. To rank up, all that should be required is to win more than you lose, with performance accounted for to make sure that you’re not just getting a buddy to hard carry you.
To lose rank, similarly, should happen naturally if you aren’t able to keep up. It shouldn’t get to the point where you could be literally afk and still (sometimes/often, depending how close you are to the protected tier)win the game because on average your teammates are better than your enemies.
You have no idea what a smurf is , do you?
A smurf is what happens when you actively throw in order to get put into easy lobbies. Smurfs don’t care about gaining rank, they generally want it to stay as low as possible. They usually have the exact opposite of the problem I described, as most games have systems in place to try to stop people from deranking for this exact purpose.
They aren’t doing it on purpose, they’re getting carried by their teammates because the game doesn’t want them to lose too often.
Yeah, that’s another part of the problem.
Pay attention dude. Are you ready? I’m only going to say this once.
It’s because the system gives them trash teammates to try to balance out their skill to create a game it thinks is a 50/50. Remember the 2200 who got a bronze teammate along with a plat to go up against a full dia/onyx lineup? Yeah. That. Should. NEVER. Happen.
Then display my mmr and call it my rank.
I imagine most people do. And yet they don’t like the rank system. I wonder why?
Woah, that other guy wrote an essay. Just read it. Respond to him, I guess.
I was just curious if you actually have a citation that win/loss affects MMR and not just defines your parameters of CSR gain/loss. From all observable experience, it’s just personal performance that matters for MMR. My ranking experience has been more successful since I started ignoring objectives and playing for slay. Regardless of win or loss, your MMR will rise if you perform sufficiently well (and won’t rise if you don’t play well). If you play great and lose, you’re just getting the opposite of what we call “CSR debt” (“CSR credit?”), as your CSR will decline -1 and then your next win will blast your CSR forward.
The assumption that CSR and MMR are generally close after sufficient matches is demonstrably not true in my experience, which we’ve walked through with game-by-game evidence in other threads.
The GDC talks talk by Menke (?2016) and also the one by Izquierdo (?2017) also talk about the curves. I particularly like how the latter talk describes them as “slimes”.
Guassian, Bayesian. My maths classes were so long ago I won’t pretend to remember the differences. The latter is more to do with doing maths on the curves?
But the point is still that it’s all about those sweet curves.
It looked to me that your skill is a curve… and all the maths done upon your skill (pre-match making and then carving up the changes in rank) are all using the curves - not a hard number.
I also found this discussion enlightening (I just wish I had a better background to follow it); https://www.moserware.com/assets/computing-your-skill/The%20Math%20Behind%20TrueSkill.pdf
The “rating” they derive from the curve to translate to a CSR is arbitrary twice; first which part of the curve to use, and secondly the units of the x-axis of the CSR.
This bit I can at least understand. Sigma is the standard deviation. 3x standard deviations is ~98% of the population.
It’s quoted a few times in the publications / discussions. Menke talked about it on the old waypoint. And as luck would have it - it’s popped up on my feed when I googled the GDC talk from 2017;
And I quote; “TrueSkill visible ranking = mu - (3 x sigma). This is a very conservative value. Actual skill is 98% likely to be higher”. He goes on to discuss how it becomes more accurate as the slimes change shape over time.
It’s not “THE” Elo system. But the principles are the same. I don’t think anyone blinks if and when it’s described as Elo or Elo-based.
I guess the 3 sigma is courtesy of the fact that the system was devised by mathematicians. If you have the capacity for extra accuracy why not go for it. Plus the division into six sigma divisions fits with the six ranking divisions (Bronze to Onyx) - but that may have just been a coincidence.
I guess I was being a bit more utilitarian. Everyone has to “carry” their own weight. Sometimes it may even be the lowest player who performs above average (maybe even just dying a few times less) that gets you over the line.
They’ve talked about this at GDC. How if the system is too “perfect” it locks everyone into a rank. Sure, the populations graphs are pristine - but it’s not necessarily everyone’s correct rank. They built margins (eg. 45-55% games) so that your rank can breathe. But as you find your place the numbers will begin to approach neutrality.
Sounds like you have read the discussion paper. Essentially TrueSkill was W/L. The discussion of TrueSkill2 was all about what weightings helped them mimic those findings faster. Of course we don’t know what values 343 eventually added to these weightings and/or how they may have ended up overstepping their bounds.
A ranking system has to take into account the relative strengths of the two teams.
It just has to.
No every win (or loss) is equal.
On a well populated server it’s probably ok for the mid level players. But for the better players (and especially better squads) there are going to be more easy games then hard games. You can’t give MMR for every win.
And the reverse will hold true for the opposite.
Sorry. You were just talking about throwing games to get easier opponents.
Ok. Paying attention, I promise.
The system isn’t giving you trash team-mates. It’s just giving you team-mates.
I guess the definition of “trash” is if you win or not. As you approach your skill ceiling you will win less games. So I guess, yes, your “team-mates” become trash. But I assume everyone on the losing team is thinking the same thing.
The bronze player was obviously squadded with someone. They made the choice. I agree it should never happen but whenever you suggest a limit of squads everyone gets up in arms about their rights to play with mates.
But you still have to take that MMR curve, choose an arbitrary point on the curve (the mean or however many sigma values away), and convert it to an arbitrary scale (eg. 1 to 50).
No citation per se. Just the papers, GDC talks, and the TS2 discussion paper.
We know that TS1 was more ELO like - it was W/L.
We know that the principle of TS2 was to identify factors that predicted the endpoint faster and more accurately. They specifically talk about things like kill rates being predictive but other combinations of kills and deaths not being any more predictive (if you have a high k/d you probably won the game).
We know that CSR is pretty much plus or minus 15 for the win and loss respectively. And that it’s amended with some of the weightings. So, at least in Slayer games your k/d (or probably k/min) help to minimise the change in CSR.
How this correlates to the change in MMR we don’t know.
They can definitely move at different rates (or even directions). Everyone will attest to the loss that drops your CSR by more than you expect. It also happens sometimes with a win and rise in rank - but nobody ever rushes to the forums to complain about that.
I think a lot of this comes down to the values that people put on a handful of CSR points.
I don’t think the system can every be accurate enough to discern between a handful of points. But people can lose their mind when a team-mate goes up by a couple more points than they did.
And the same when CSR and MMR drift a bit. What’s reasonable? A few points? 10? 25? When are they no longer the same thing?
Your CSR is a reflection of your MMR. It’s not that hidden.
If your MMR curve suggests your rank is 900 to 1000 (98% probability) then your CSR will be at (or around) 900. Platinum 1.
What I think then happens (and I need someone better at maths then me) is that the ranking system tends to use the whole width of your curve. So your average opponent will be closer to the mean of your curve (950). They, of course, will all have their own curves of different widths (as well as everyone else in the game) - but the overlap of each curve will balance out.
It’s not that they are out to get you, it’s that evidently the ranks do not correlate especially strongly with mmr. Plenty of people have noticed this, it’s a major reason so many people are complaining about the rank system, and why 343 felt the need to put out a blog post on the subject.
The curve displayed on that page is a graphic to show what the numbers mu and sigma show. Literally on that page they state that they are using the values of mu and sigma.
So they are using mu, the number, as your mmr at any given time. They get that number by taking the mean of all past skill values generated. That’s the skill distribution, the curve, but they use it to get a single number, which is used to match you. They also note that tracking sigma is just something they do to help them adjust the mu value. to narrow in on an expected true value.
Point is, your mmr is not a curve, it’s a number.
A Gaussian distribution is a normal distribution, also called a bell curve. Bayesian inference is basically using the Bayes theorem to update the probability of an hypothesis as new data are gathered.
Generated by finding a best fir to a set of numbers, and used to output a number for use, but yes.
Evidently quite a lot has changed since 2011.
Is this on trueskill 1 or 2?
It really doesn’t fee like it. It feels like a combination of complete randomness, expected from matching with a fluctuating mmr number, and an attempt at laser fine balance based on recent performance.
A lot of the problem is they’re trying to be too perfect and fast with it. You want the system to be a bit slow in finding the appropriate balance so that players can feel it when they improve, and they can get the wins they need to move their rank to the correct region. Which can be easily done by using the visible number to matchmake and the mmr number to accelerate it in the right direction.
As is, if you improve you will never feel it, because you won’t get the feedback of a temporarily improved winrate (coinciding with increasing rank to make sure you notice) with the system finding the “perfect” balance so fast.
You don’t have to. The point of the mmr is to be an invisible weight to pull your rank in the right direction. It is meant to resist you getting a buddy to carry you, block derankers, all that stuff. That doesn’t mean you should rank up for losing games on average.
If you win more than you lose on average, you are probably better than your rank, and so should rank up overall until this is no longer the case. If you aren’t winning on average, you probably shouldn’t be ranking up. Then mmr corrects for stuff like having a buddy boost you, since your performance in harder lobbies suggests you don’t belong there. But if you aren’t losing games, it would be unfair to derank you for winning, because the stats can be wrong.
It all comes back to the same core design flaw of not matching based on rank.
No, I was talking about what I expected would happen to players at the bottom of the skill curve based on what we know happens at the top end.
If your mmr is high enough, it’s giving you teammates with reduced mmr (expected skill) compared to the enemy team in order to keep the mmr averages of each team roughly equivalent.
You know damn well what I mean.
Realize that I’m not good enough at the game for this to be a real problem for me. I’m outlining the problems caused by this system.
Maybe, but he still shouldn’t have been in that lobby. The 2200 was streaming, so I know he was in solo, and we had 2 dia/onyx duos. Even if the bronze was queued in with the 1500, they should not have been on that team. Much less with a plat to round it out.
Sure, how about the mean value that they already use to matchmake. And they can just keep whatever scale they already have it on.
On a mature account, say 40+ games post placement, they seem to correlate pretty well. And I can see no logical reason why they wouldn’t.
Plenty of people have no idea that they behave as a curve. I imagine lots of the “complaints” are post placement when your CSR is definitely lower than your MMR (three times sigma).
Also people aren’t taking into account weightings for squads and form when they are trying to square off the teams.
Now that you know that your CSR is 3x sigma below the mean - and that early in the season this may be quite different to the mean - does it change your perception of the match ups in any way?
A lot of people are also complaining about not having a concrete MMR gain for every (and any) win. Not a convincing cohort.
It was overdue. And hopefully just the start. I have so many questions!
It’s an introductory page. They also use phrases like “belief in your skill using these two numbers”, and terms like “uncertainty”.
The GDC talks also go about “uncertainty”.
And Menke, either in his GDC talk or on Waypoint, definitely said that the TrueSkill algorithm itself never puts a hard value on your skill.
And if you think about it. When you start your curve is low and flat - the mean doesn’t mean (he he) a lot. In regards to it acting as a probability curve there is a lot of area under the curve to the sides of the mean.
Your MMR is a curve. The maths done by the TrueSkill algorithm involve the comparison and manipulation of curves. The width of your curve affects your match-making and the volatility of your rank. You can describe it with a number, eg, the mean, but if when you have to describe it with two numbers (reflecting the shape of the curve), it’s a curve.
Two different players with the same mean but different sigma values will behave very differently on the result of the game.
And interestingly. I just noticed on that page, in the FAQ section, they actually mention the -3xsigma as the “value” for your TrueSkill.
The main thing is they changed the units of the x-axis. 1-50 worked so much better than 1-1800+. In practice the accuracy system doesn’t seem to function at that level.
I guess the citations are for TS1. But I’ve never seen it described any differently for TS2. Including all the posts by Josh Menke on the old Waypoint. I do remember him actually talking about it in regards to Halo 5 - but that’s all lost to the ether now.
The reality with modern gamers is that you will find and reach your skill ceiling pretty quick. We’ve all played a lot of Halo and/or other FPS. There is no such thing as gradual journey of discovery etc.
We are what we are and if you put an effort in you will max out pretty quickly - and any improvement from there could take weeks and/or months (if at all).
If the system can work you out in a handful of games. So be it.
Close. If you win more than you lose on average against teams ranked better than you, you are probably better.
The truth is match-making is not going to find 50:50 games (or around there) for everyone. The top handful of teams on any one server are going to struggles - unless they want to play each other over and over and over.
From what we’ve seen in other threads there are plenty of high level players with W/L greater than 50% - but a rank that doesn’t go up. Their CSR tends to drift up but comes back on the next loss (invariably a 3v4 just to rub salt into the wound).
Isn’t this the same thing. Match-making is struggling to find you team-mates and opponents of similar level. So it has to range out a bit. I’m guessing the big fish on the other team is feeling the same struggle.
I agree. Whole heartedly. But there are a lot of people who want to play with mates. The silly things is they usually then go onto complain that their mates get beaten up and they spend the whole match trying to protect their mate AND carry the team.
Bottom line is there should be a range limit of squads in ranked.
When your curve is wide the mean is a dangerous precedent. There is a good chance your skill is below this - and taking away someone’s rank hurts more than lying to them a bit at the beginning. I think they used the term “protecting the player” in one of the talks.
I don’t know why they chose -3 x sigma per se. It is, as they said in the paper, conservative.
But as the curve narrows, sigma gets very small, and -3x sigma is pretty much your mean anyway. Just as long as they don’t let it get Halo 3 type small.
I won’t quote cause these strings are huge. But Darwi, Ken and I pretty much established the only way to explain my sequence of games was drift of over 100 points (such that winning 80% of my next 25 games resulted in over 100 loss of CSR).
Ken noted that my MMR was likely far below my CSR for whatever reason, so the system effectively ignored my wins (by giving plus one CSR) while dropping my MMR behind the scenes each match based on individual performance, until it had a “catch up chance” (re: a loss) when it then removed the max possible CSR.
So that’s the magnitude of difference between CSR and MMR we’re talking about in fully mature accounts. And just because other people don’t take the time to actually hash it out in the data doesn’t mean they aren’t experiencing similar issues.