Matchmaking Feedback Update – March 13

Games Other Halo
1

When the smurfing improvements get applied, will I get penalized for playing with my friends who are legitimately worse than me?
No. Other than the fact that your friends will have pretty rough games because it will still match you towards the middle. So if you’re Onyx and your friends are Gold, your friends will be getting stomped by Diamonds.

Smurfing because I can’t find matches in Onyx
This is understandable. Unfortunately, if enough people do this, then it becomes a reinforcing problem. There aren’t enough people in Onyx because they’re all smurfing, which causes more to smurf.
We’re working to mitigate this so high-ranked players can match more consistently, so stay tuned.

Isn’t the system broken if players have a win % of 65-66?
No, that’s normal in all the best matchmaking systems because of:

  • Noise - Imperfections in the model - Great, new players that the system is slow to adapt to because of noise - Parties not handled efficiently yet - A lack of competition during the time that the player is searching. Sometimes intentional on the part of the player, combined with a need to not make players wait forever
    Questions about Solo Queue or Party Matching
    If you read back into previous posts I talk about how we are planning to address this. We are first going to try a more efficient approach than hard cut-offs because those hurt the population. We’re developing a system that knows how much better a party is than a non-party, and will matchmaking accordingly. If the only fair opponent is another party, it will do that.

A solo queue would be a bad idea at this point because:

  • If a solo only playlist is 100% successful and all solo players go to the solo playlist, then parties of 3 can no longer play the game. At all. In addition, all other party sizes wait much much longer for a match because there would be way less people to play. - If a solo playlist is less than 100% successful, then everyone suffers and waits longer than before because now there are less people in both playlists.My experience with solo only queues in FPS is they cause more harm than good.

On the other hand, I think we can give more fair matches than we do today to solo players without needing a separate playlist, but it’s going to take a bit of time to implement. I will keep you posted. The same system that will help with party matching will also help with smurfs, so those solutions will come together in roughly the same time frame.

CSR Changes Update
After placement matches, we start your CSR behind your expected CSR (MMR) because:

  • It might have overestimated you and we prefer you go up rather than down if it is wrong. - We want you to “prove” you deserve the expect CSR. Just like in, e.g., NCAA sports. The ranking systems can all say a team is #1, but they still have to win the tournament, or win the bowl, to be crowned. The system may expect you to be Diamond, but wants you to first play like one for several games in the season to earn it.
    In previous seasons, CSR started about 30-50 games behind expected. This season it is about half that (15-25 games), so it should take less time to “prove” you deserve what MMR expected of you. In CSR and MMR terms, CSR was set 200 rating behind your MMR after placement. Now it is 100. So a true 1650 Onyx player will start at 1550, for example. A true 1550 Onyx player will start in Diamond 6.
    For Onyx, we haven’t changed the CSR boundary, but instead we made a higher MMR map to 1500. After placement, some players will be farther from Onyx than they used to be, and probably won’t get there. Those that still deserve Onyx after this change should have no problem getting there within roughly 15-25 games.

There was some confusion around just how high CSR can be out of placement. In general, it is always 100 back from your MMR. This means you can indeed place into Onyx.
The exception is high-Onyx players will never place higher than Onyx 1700, and you need at least 1800 to be Champion. This was to prevent people from getting Champion by just starting out earlier in the season than everyone else. This is also partly how we caught the problem with this season’s first reset. People were Champ right out of placement.
You may wonder why you can go 10-0 in placement and not get into Onyx. This is because either:

  • You are brand new and the system hasn’t seen enough past history to believe you are Onyx yet. This also means you probably went 10-0 vs. mostly Gold and and Platinum players, which isn’t convincing enough to make you Onyx. - You have an established history already, and the system is trusting that history over your recent performance - In addition, if you have a past history of being, e.g., Silver, your 10 matches will be mostly against Silver, and 10-0 vs. Silver does not make you Onyx.

Social vs. Ranked
In general, I agree with eLantern’s philosophy on social vs. ranked. That said, there’s this funny paradox that occurs between Ranked and Social which I’ve come to accept over the years:

  • Competitive players play social to relax. They expect looser matches so they don’t have to try as hard. - Casual players (especially lower-skilled ones) get the opposite experience. They get owned in Social, and have the impression that Ranked is the only fair place for them. That, or they just stop playing the game because they may think Ranked is going to be even worse. - When Casuals get driven to Ranked, this ironically annoys competitive players who sometimes get “casuals” on their teams.This is a tricky problem to solve. I personally think Social should have a mix of tight and loose matches, treating the worst Casuals carefully so as not to lose them and keep the population healthy. Even Ranked could benefit from the occasional looser match to add to the variety.

Some of the community have suggested we increase the Social offerings to get casuals out of Ranked, but I don’t think Casuals are in Ranked because of a lack of offerings. I think they’re there because the perception is that’s where fair matches are.

So, ironically, Social ends up being where the competitive players go to chill and play mostly vs. each other, and Ranked ends up being where Casuals go almost all the time for more fair matches, and Ranked players go when they want to “lean forward”

CSR vs. MMR
A few more suggestions were given on how to communicate the matchmaking choice vs. the visible CSR. We’re listening to those, and looking at a number of ways to make this more clear, but we won’t see this in Halo 5.

I think it would be cool to tell players their exact performance after a match in CSR terms. For example, a Gold player might see “even though you’re Gold, you just played like a Diamond player, so don’t be surprised if you see Diamond opponents! Keep it up and you’ll get to Diamond!” Stuff like that.

Challenges
A few of you have asked about having some set of daily goals to hit in the form of “quests”. Halo 4 had this in the form of rotating Challenges. We also like this feature and would love to improve on it even further, though we won’t be seeing it in Halo 5.

2

Are some Game Modes harder to Rank, and so you leave them out of Ranked?
No, this is not a driving reason behind why we wouldn’t Rank something, but it is a very interesting point to bring up.

Some game modes do actually take less skill than others, and this does show up if we analyze the ranks. The core idea is if we have a near perfect estimate of everyone’s skill, and the outcome of a match is still somewhat random, that means skill isn’t what determines who wins. Instead it is just random.

Game Modes where it is highly random who wins aren’t skillful. So, in theory, we could use that information to throw modes out of Ranked. We don’t actually do that, but it does so happen that modes that are less skillful tend to not be as popular in Ranked anyways, and for that reason end up washing out for that reason instead.

3

Your updates are easily my latest favorite thing on Waypoint.

Thanks for all the information! Definitely curious to see what you’ve got going on for smurfing. :slight_smile:

4

> 2533274839818445;1:
> CSR vs. MMRA few more suggestions were given on how to communicate the matchmaking choice vs. the visible CSR. We’re listening to those, and looking at a number of ways to make this more clear, but we won’t see this in Halo 5.
>
> I think it would be cool to tell players their exact performance after a match in CSR terms. For example, a Gold player might see “even though you’re Gold, you just played like a Diamond player, so don’t be surprised if you see Diamond opponents! Keep it up and you’ll get to Diamond!” Stuff like that.

That sounds a lot like the system Halo: Reach had. Correct me if I am wrong, isn’t that what the player rating stat meant after a match in Reach? I agree that would be a fun thing to see in a PGCR.

5

> 2533274959866431;4:
> That sounds a lot like the system Halo: Reach had. Correct me if I am wrong, isn’t that what the player rating stat meant after a match in Reach? I agree that would be a fun thing to see in a PGCR.

In this case, I meant the performance of the entire team.

You could also have a personal performance after each match, although we would calculate the performance differently than Reach did it.

6

> 2533274839818445;1:
> Questions about Solo Queue or Party MatchingIf you read back into previous posts I talk about how we are planning to address this. We are first going to try a more efficient approach than hard cut-offs because those hurt the population. We’re developing a system that knows how much better a party is than a non-party, and will matchmaking accordingly. If the only fair opponent is another party, it will do that.
>
> A solo queue would be a bad idea at this point because:
> - If a solo only playlist is 100% successful and all solo players go to the solo playlist, then parties of 3 can no longer play the game. At all. In addition, all other party sizes wait much much longer for a match because there would be way less people to play. - If a solo playlist is less than 100% successful, then everyone suffers and waits longer than before because now there are less people in both playlists.My experience with solo only queues in FPS is they cause more harm than good.
>
> On the other hand, I think we can give more fair matches than we do today to solo players without needing a separate playlist, but it’s going to take a bit of time to implement. I will keep you posted. The same system that will help with party matching will also help with smurfs, so those solutions will come together in roughly the same time frame.

What if there were options to solo q or to have looser matches similar to the connection preferences already in place. It allows teams of 3 to play and lets people who want to only play solo play solo.

7

> 2533274912317447;6:
> What if there were options to solo q or to have looser matches similar to the connection preferences already in place. It allows teams of 3 to play and lets people who want to only play solo play solo.

You still have the same problems if everyone chooses solo.

There are much more efficient solutions.

8

> 2533274839818445;7:
> > 2533274912317447;6:
> > What if there were options to solo q or to have looser matches similar to the connection preferences already in place. It allows teams of 3 to play and lets people who want to only play solo play solo.
>
> You still have the same problems if everyone chooses solo.
>
> There are much more efficient solutions.

There will always be people who will choose the faster option. That’s how it works with the current options. I can’t speak for other people but I personally just don’t want to play with teams of 2 or 3 on my team who are usually in party chat. In the system you talked about that can still happen if it feels like the teams are even.

9

Will we ever be seeing a classic playlist in Halo 5, with sprinting and abilities turned off?

I understand that it won’t be the most balanced thing in the world, but having it is still better than not having it. It is literally the only thing that will get me to ever re-install Halo 5.

10

> 2811398874529013;9:
> Will we ever be seeing a classic playlist in Halo 5, with sprinting and abilities turned off?
>
> I understand that it won’t be the most balanced thing in the world, but having it is still better than not having it. It is literally the only thing that will get me to ever re-install Halo 5.

I wouldn’t hold your breath. If you really want to play classic settings you could probably find customs in the custom browser or set one up yourself I’m sure the lobby would fill up.

11

I have a bit to add that I meant to put in the original post:

Why hasn’t party imbalance been addressed up till now?

This is a fair question. There are a number of reasons, many of which are beyond my area of expertise and relate to general game production, and I won’t go into those. Just know that you can’t always get the features you want into a game when you want them. Game development requires a lot of patience and balancing with many parts of the team, and hoping things turn out the way you want. For a little personal history, I left 343 a year before the Halo 5 shipped, and recently returned. When I left, I didn’t know how many of my features still in flight would get still get done. From past experience, I knew to only expect a handful of them to make it in. Well, almost all of them got in, which is more than you can usually get in game development. I was impressed with how well features I had designed got done without me there. Really impressed.

So back on the topic. We do a thorough analysis of any proposed major changes or new features because they impact millions of players. In the case of party matchmaking, we’ve been testing solutions to this problem for years using the millions and millions of real matches that we have from several Halo games, and we’ve found major problems with almost every solution. Problems would negatively impact the majority of the population. Until now, we haven’t had a solution we felt fair enough to all the players. We now think we have a good one in the works.

For an example of the type of analysis we do, we looked at the win percentages of players who play in all different party sizes, including those who play solo. You would expect that players who play in parties always win more.

This is wrong.

In fact, in real data, if you were to look at what happens when the majority of our population plays in a party of 4, they lose more in parties than when solo. You may have a hard time believing this, and so did we, so we dug deeper.

We found some interesting things:

  • By far, most players don’t play in parties - Players who aren’t used to playing in parties actually play much worse when they party up than when they play solo. They win less. - The much smaller group of players who commonly play in parties do play much better than solo playersWhich means, yes, we need to address parties vs. solo, but we need to be very careful in how we do it. We need to make sure the players who are actually worse in parties aren’t given crazy hard matches and get destroyed by those who always play in parties.

Because according to this data, if I just take, any e.g. to4 and give them harder matchmaking, I will be affecting millions of players who are just occasionally partying up with friends, and they will get owned. This is what I’ve seen happen in many games:

  • Friends are online for once! Sweet, let’s party up! - Matchmaker gives us a much harder match since we’re in a party (either forces vs. another to4, or 4 super good solo players) - Whoah, we got owned - This game isn’t as fun with friends - Guess I’ll play solo - Eventually wash outThis is not acceptable. Playing with friends should never be worse than playing solo. Playing with friends should keep you playing. This situation represents the majority of our players by far. So we can’t just do what may seem obvious.

The solution we are putting in can account for these differences, and make sure in the majority case where casual friends group up, they get a fair, fun match, and don’t get owned.

Likewise, when a true to4 parties up, it will make sure they only play either another to4 of similar skill, or much much better solo players whose skill makes up for the organization of the to4. In the case where it’s not possible to find a fair group of solo players, it will only match vs. to4.

12

Thank you for these updates, they give me hope that Halo 5 multiplayer will actually be enjoyable again.

13

> 2533274839818445;11:
> I have a bit to add that I meant to put in the original post:
>
> **Why hasn’t party imbalance been addressed up till now?**This is a fair question. There are a number of reasons, many of which are beyond my area of expertise and relate to general game production, and I won’t go into those. Just know that you can’t always get the features you want into a game when you want them. Game development requires a lot of patience and balancing with many parts of the team, and hoping things turn out the way you want. For a little personal history, I left 343 a year before the Halo 5 shipped, and recently returned. When I left, I didn’t know how many of my features still in flight would get still get done. From past experience, I knew to only expect a handful of them to make it in. Well, almost all of them got in, which is more than you can usually get in game development. I was impressed with how well features I had designed got done without me there. Really impressed.
>
> So back on the topic. We do a thorough analysis of any proposed major changes or new features because they impact millions of players. In the case of party matchmaking, we’ve been testing solutions to this problem for years using the millions and millions of real matches that we have from several Halo games, and we’ve found major problems with almost every solution. Problems would negatively impact the majority of the population. Until now, we haven’t had a solution we felt fair enough to all the players. We now think we have a good one in the works.
>
> For an example of the type of analysis we do, we looked at the win percentages of players who play in all different party sizes, including those who play solo. You would expect that players who play in parties always win more.
>
> This is wrong.
>
> In fact, in real data, if you were to look at what happens when the majority of our population plays in a party of 4, they lose more in parties than when solo. You may have a hard time believing this, and so did we, so we dug deeper.
>
> We found some interesting things:
> - By far, most players don’t play in parties - Players who aren’t used to playing in parties actually play much worse when they party up than when they play solo. They win less. - The much smaller group of players who commonly play in parties do play much better than solo playersWhich means, yes, we need to address parties vs. solo, but we need to be very careful in how we do it. We need to make sure the players who are actually worse in parties aren’t given crazy hard matches and get destroyed by those who always play in parties.
>
> Because according to this data, if I just take, any e.g. to4 and give them harder matchmaking, I will be affecting millions of players who are just occasionally partying up with friends, and they will get owned. This is what I’ve seen happen in many games:
> - Friends are online for once! Sweet, let’s party up! - Matchmaker gives us a much harder match since we’re in a party (either forces vs. another to4, or 4 super good solo players) - Whoah, we got owned - This game isn’t as fun with friends - Guess I’ll play solo - Eventually wash outThis is not acceptable. Playing with friends should never be worse than playing solo. Playing with friends should keep you playing. This situation represents the majority of our players by far. So we can’t just do what may seem obvious.
>
> The solution we are putting in can account for these differences, and make sure in the majority case where casual friends group up, they get a fair, fun match, and don’t get owned.
>
> Likewise, when a true to4 parties up, it will make sure they only play either another to4 of similar skill, or much much better solo players whose skill makes up for the organization of the to4. In the case where it’s not possible to find a fair group of solo players, it will only match vs. to4.

Does this hold up when you only account for Onyx+ ranks? My own personal experiences of playing on smurf accounts lines up pretty well with this where in the Plat-Diamond range it’s like 95% solo players and most teams are either just some buddies not really trying or aren’t good enough to really have any advantage.

In Onyx-Champion I feel like there are a lot more teams and they almost always have the edge over solo players.

14

I’ve been frustrated by the win/loss only determination of rank. If I am on a team as a solo or maybe party of two (99% of my games) and I do well but the others on the team do not then why can’t I get at least a small bump in ranking provided I played well.

This way when the system matches players the good players still float to the top the bad ones to the bottom and maybe we will get some more fair matches? If this is how it works in the backend but just not on the visible rank then can we fix that to reward players who play better but may not be on a team of individuals who caused them to lose the match?

15

> 2533274912317447;13:
> Does this hold up when you only account for Onyx+ ranks? My own personal experiences of playing on smurf accounts lines up pretty well with this where in the Plat-Diamond range it’s like 95% solo players and most teams are either just some buddies not really trying or aren’t good enough to really have any advantage.
>
> In Onyx-Champion I feel like there are a lot more teams and they almost always have the edge over solo players.

I can believe that. So that could lead to an alternate hack solution that only gives harder matches to Onyx+ parties, which I think would kind of work.

This is where some tough game dev choices come in though. Should we build a throwaway solution while waiting for the main one to finish? The answer to that in most development teams is going to be “please no”

The incoming solution will also do this, and also addresses a number of other points that could use some polish, so rather than slow that solution down to try a hack, we usually prefer doing it “right” the first time.

But, yes, that’s a great observation.

16

Is there any chance that we could see ricochet or oddball but into the game under the social playlist ? I’ve been dying for a definitive “maybe” or “heck no”

17

> 2533274810961906;14:
> I’ve been frustrated by the win/loss only determination of rank. If I am on a team as a solo or maybe party of two (99% of my games) and I do well but the others on the team do not then why can’t I get at least a small bump in ranking provided I played well.
>
> This way when the system matches players the good players still float to the top the bad ones to the bottom and maybe we will get some more fair matches? If this is how it works in the backend but just not on the visible rank then can we fix that to reward players who play better but may not be on a team of individuals who caused them to lose the match?

Yes, I agree. It’s not something we can fairly represent in Halo 5, but my personal design philosophy agrees with you that we should have a way to both:

  • Make it clear that winning as a team is still the highest bar to achieve. - Reward and account for great individual performances, even when you lose
18

> 2533274839818445;1:
> CSR Changes UpdateAfter placement matches, we start your CSR behind your expected CSR (MMR) because:
> - It might have overestimated you and we prefer you go up rather than down if it is wrong. - We want you to “prove” you deserve the expect CSR. Just like in, e.g., NCAA sports. The ranking systems can all say a team is #1, but they still have to win the tournament, or win the bowl, to be crowned. The system may expect you to be Diamond, but wants you to first play like one for several games in the season to earn it. In previous seasons, CSR started about 30-50 games behind expected. This season it is about half that (15-25 games), so it should take less time to “prove” you deserve what MMR expected of you. In CSR and MMR terms, CSR was set 200 rating behind your MMR after placement. Now it is 100. So a true 1650 Onyx player will start at 1550, for example. A true 1550 Onyx player will start in Diamond 6.
> For Onyx, we haven’t changed the CSR boundary, but instead we made a higher MMR map to 1500. After placement, some players will be farther from Onyx than they used to be, and probably won’t get there. Those that still deserve Onyx after this change should have no problem getting there within roughly 15-25 games.
> There was some confusion around just how high CSR can be out of placement. In general, it is always 100 back from your MMR. This means you can indeed place into Onyx.
> The exception is high-Onyx players will never place higher than Onyx 1700, and you need at least 1800 to be Champion. This was to prevent people from getting Champion by just starting out earlier in the season than everyone else. This is also partly how we caught the problem with this season’s first reset. People were Champ right out of placement.
> You may wonder why you can go 10-0 in placement and not get into Onyx. This is because either:
> - You are brand new and the system hasn’t seen enough past history to believe you are Onyx yet. This also means you probably went 10-0 vs. mostly Gold and and Platinum players, which isn’t convincing enough to make you Onyx. - You have an established history already, and the system is trusting that history over your recent performance - In addition, if you have a past history of being, e.g., Silver, your 10 matches will be mostly against Silver, and 10-0 vs. Silver does not make you Onyx. CSR vs. MMRA few more suggestions were given on how to communicate the matchmaking choice vs. the visible CSR. We’re listening to those, and looking at a number of ways to make this more clear, but we won’t see this in Halo 5.
>
> I think it would be cool to tell players their exact performance after a match in CSR terms. For example, a Gold player might see “even though you’re Gold, you just played like a Diamond player, so don’t be surprised if you see Diamond opponents! Keep it up and you’ll get to Diamond!” Stuff like that.
>
>
> I can confirm that this is true, played 10 matches of slayer and lost quiet average and i got on platinum 1, which i was in the previous season. based on what i read it sounds like the matchmaking system works kinda like a gambling machine in this scenario. for example, you stated and i quote
> - The system may expect you to be Diamond, but wants you to first play like one for several games in the season to earn it.Im stuck on platinum 1 due to solo queue and i admit thats part of a players fault, but based on what i understand on “proving your rank” in which my case is platinum 1 means i will be forever platinum or have a chance on moving forward? i ask this because when i play solo i do get matched with high diamonds but thats because you state that a gold player plays just as good as a diamond. moving on, is the system supposed to make a team with similar skills based on my experience in solo or does that apply only on full teams? this question comes to mind because i had teammates who are actually really bad considering that i do well on my own most of the time. so proving my rank will stay the same because thats where i belong. will the system drop me on harder matches to lose rank just to stay in my current rank? this comes to the fact that some times i get matched against really weak or unskilled opponents, thus making it an instant victory but other times i get really hard matches to the point of my own teams quits and leaves me alone.
>
> i understand that this is a risk im taking for playing solo but just wanted to clarify my “proven rank” so does this means this is where i belong as a solo player?

19

> 2533274839818445;17:
> > 2533274810961906;14:
> > I’ve been frustrated by the win/loss only determination of rank. If I am on a team as a solo or maybe party of two (99% of my games) and I do well but the others on the team do not then why can’t I get at least a small bump in ranking provided I played well.
> >
> > This way when the system matches players the good players still float to the top the bad ones to the bottom and maybe we will get some more fair matches? If this is how it works in the backend but just not on the visible rank then can we fix that to reward players who play better but may not be on a team of individuals who caused them to lose the match?
>
> Yes, I agree. It’s not something we can fairly represent in Halo 5, but my personal design philosophy agrees with you that we should have a way to both:
> - Make it clear that winning as a team is still the highest bar to achieve. - <mark>Reward and account for great individual performances, even when you lose</mark>

How do you balance this appropriately? Like, how do you balance rewarding players on the losing team that do well but keep them from going for personal stats at the cost of the team objective?

20

> 2533274839818445;15:
> > 2533274912317447;13:
> > Does this hold up when you only account for Onyx+ ranks? My own personal experiences of playing on smurf accounts lines up pretty well with this where in the Plat-Diamond range it’s like 95% solo players and most teams are either just some buddies not really trying or aren’t good enough to really have any advantage.
> >
> > In Onyx-Champion I feel like there are a lot more teams and they almost always have the edge over solo players.
>
> I can believe that. So that could lead to an alternate hack solution that only gives harder matches to Onyx+ parties, which I think would kind of work.
>
> This is where some tough game dev choices come in though. Should we build a throwaway solution while waiting for the main one to finish? The answer to that in most development teams is going to be “please no”
>
> The incoming solution will also do this, and also addresses a number of other points that could use some polish, so rather than slow that solution down to try a hack, we usually prefer doing it “right” the first time.
>
> But, yes, that’s a great observation.

Is it possible or practical to scale it so higher ranks get stricter matches?