Matchmaking Feedback Update – June 19

Games Other Halo
1

No big updates this week. Still cranking away at features I’ve brought up in the past.

Instead, here are some musings of mine on using data to look at playlist health.

Playlist Health

Here are two ways to look at Playlist health that I think are useful.

  • Can players find matches.
    This is the simplest one. If players can’t even find matches in a playlist, it’s definitely time for some form of maintenance.

  • Can players find good matches.
    In some cases, it’s not just whether we have enough players playing to make matches in those playlists, it’s whether we have enough players to make matches that are fair. Playlists tend to struggle as their level of fairness decreases.

One simple measure of a single match’s UN-fairness is how predictable its outcome is. If it’s obvious beforehand that Blue team will win, then the match isn’t that fair, it’s stacked in favor of the Blue team.

If most of the matches on a playlist have this problem — that one team is consistently more likely to win—then the entire playlist isn’t that fair, and a candidate for maintenance.

One value that is good at measuring predictability is how often the team with the highest MMR (skill rating) wins a match. Our skill system is pretty good at finding play skill, so it should be good at predicting match outcomes if the skill gap is too wide.

A perfectly fair playlist would be all matches where the team with the highest MMR wins only 50% (half) of the time. A playlist becomes less and less fair as the accuracy of the skill system at predicting that outcome increases. A playlist where we can predict the winning team correctly, e.g., 75% of the time is not a fair playlist.

This amount can vary greatly depending on the game modes in the playlist and its population. Different game modes have different levels of sensitivity to the skill gap.

If a game mode is less sensitive to a wide skill gap, it can support a smaller population and still have fair matchmaking. Playlists with modes like that are easy to maintain.

On the other hand, a game mode that is sensitive to the skill gap needs a larger population to provide the same quality of matchmaking and can fall under scrutiny even though it may seem to have plenty of players. Playlists with modes like that are harder to maintain, and may require the occasional “tough call” to keep the rest of the ecosystem healthy without falling below a reasonable quality bar.

Hopefully some fun thoughts for this week.

2

Who should one pester at 343 about creating a position called “Menke’s Minion” so someone can just sit and absorb all the numbers and work you put into MM?

Asking for a friend.

3

So how exactly is “Fairness” calculated?
Is it just an average of Blue Team’s MMR vs. Red Team’s?

4

Definitely not a easy task finding a balance between the few, but don’t the current search preference options resolve it fairly well? If I understand it correctly focused will put you in a fair match with a higher wait time while expanded does the opposite. That being said I’m not entirely sure how these perimeters work and I would love to hear your take on the subject.

5

Can you mention which playlists have you found the most healthy and which ones need more care?

6

> 2533274796457055;2:
> Who should one pester at 343 about creating a position called “Menke’s Minion” so someone can just sit and absorb all the numbers and work you put into MM?
>
> Asking for a friend.

I volunteer as tribute

7

Sure you’d get a lot less complaining if the population was higher to make matches more balanced…wish halo wasn’t dead :confused:

8

Have you found any significant difference in predicted skill of players when they are playing Objective games versus Slayer games, in playlists that have a combination of both? Or even particular maps?

Would some form of differentiation be able to help, or would it just make the system overly-complex to the point of being less predictable?

Similarly, would some form of temporary MMR reduction for players who are in their first few matches of the day be at all beneficial?

9

Does 343 take action against players who exploit the matchmaking system to get more favourable matches or is that under Xbox live jurisdiction?
Example being 3 champs in a fireteam with a bronze

Edit:
Basically are there other things besides quitting that could earn someone a bann?

10

> 2535450479150563;9:
> Does 343 take action against players who exploit the matchmaking system to get more favourable matches or is that under Xbox live jurisdiction?
> Example being 3 champs in a fireteam with a bronze

I suggest you read through the archives and the prior threads. This is addressed several times over.

11

> 2533274796457055;10:
> > 2535450479150563;9:
> > Does 343 take action against players who exploit the matchmaking system to get more favourable matches or is that under Xbox live jurisdiction?
> > Example being 3 champs in a fireteam with a bronze
>
> I suggest you read through the archives and the prior threads. This is addressed several times over.

I think it was explained how a players CSR behaves in a situation like I mentioned. However that is not what I was getting at. I will edit my post to clarify my question.

12

> 2535450479150563;11:
> > 2533274796457055;10:
> > > 2535450479150563;9:
> > > Does 343 take action against players who exploit the matchmaking system to get more favourable matches or is that under Xbox live jurisdiction?
> > > Example being 3 champs in a fireteam with a bronze
> >
> > I suggest you read through the archives and the prior threads. This is addressed several times over.
>
> I think it was explained how a players CSR behaves in a situation like I mentioned. However that is not what I was getting at. I will edit my post to clarify my question.

What you’re suggesting is not against the XBL TOS.

13

> 2533274796457055;12:
> > 2535450479150563;11:
> > > 2533274796457055;10:
> > > > 2535450479150563;9:
> > > > Does 343 take action against players who exploit the matchmaking system to get more favourable matches or is that under Xbox live jurisdiction?
> > > > Example being 3 champs in a fireteam with a bronze
> > >
> > > I suggest you read through the archives and the prior threads. This is addressed several times over.
> >
> > I think it was explained how a players CSR behaves in a situation like I mentioned. However that is not what I was getting at. I will edit my post to clarify my question.
>
> What you’re suggesting is not against the XBL TOS.

one of the reasons i stopped playing arena. every time a new matchmaking update is up i try to bring the current state of smurfing and im always getting ignored. i ask directly to josh but it seems this is an issue that we might just start getting used to it. im not trying to be negative nor have something against 343, im just trying to voice my thoughts on something unstoppable. i been dealing with smurfs to the point i gave up arena. it almost seem very easy to manipulate the matchmaking system without getting caught. its almost forcing me to start smurfing just to get a fair match since arena is infested with smurfs. i blame myself for going solo on arena but even with a team it still happens no matter what.

i know this has been discussed so many times and im not providing constructive feedback but all im trying is to get a word from you guys.

as always, thanks josh for the updates.

14

> 2533274888695001;3:
> So how exactly is “Fairness” calculated?
> Is it just an average of Blue Team’s MMR vs. Red Team’s?

Even simpler.

Given a set of matches, count the number of times the team with the higher MMR wins.

Divide by the number of matches.

That shows how predictable that set of matches was.

If matchmaking was perfectly even, then each match would be a coinflip, and that percentage would be 50%.

If matchmaking isn’t even, then you get higher and higher values.

As it approaches 100%, the quality is really bad since the outcome is known beforehand, which isn’t fair.

15

> 2533274950162674;4:
> Definitely not a easy task finding a balance between the few, but don’t the current search preference options resolve it fairly well? If I understand it correctly focused will put you in a fair match with a higher wait time while expanded does the opposite. That being said I’m not entirely sure how these perimeters work and I would love to hear your take on the subject.

The only difference in matchmaking quality when you search Focused vs. the others is that if you don’t search Focused, someone who has waited longer than you can “steal” you and bring you into a worse quality match as long as it fits “their” current tolerance.

But there’s still a maximum allowed unfairness.

16

> 2533274834881503;8:
> Have you found any significant difference in predicted skill of players when they are playing Objective games versus Slayer games, in playlists that have a combination of both? Or even particular maps?

Skill is relatively similar mode to mode. Though the new system we are building measures skill at the mode level rather than the playlist one. Matchmaking will probably be done by weighting the average across the possible modes on the playlist since mode isn’t pre-chosen.

But it could give potentially better team balancing. Though the differences aren’t huge.

> 2533274834881503;8:
> Would some form of differentiation be able to help, or would it just make the system overly-complex to the point of being less predictable?

We look at a lot of stuff like this and incorporate whatever is helpful.

> 2533274834881503;8:
> Similarly, would some form of temporary MMR reduction for players who are in their first few matches of the day be at all beneficial?

We haven’t found this to be needed yet. The newer system does a good job at predicting your outcome your first game back of the day without it.

17

Excellent updates, as always, Dr. Menke. I always enjoy reading your thoughtfully composed, accurately written, and transparent updates.

Is there an average wait-time differential 343 has been able to garner from their data from the MM servers, in comparing “Focused” matchmaking vs “Expanded” selection? I traditionally have always chosen balance, so I’d just like an answer from 343 regarding these time figures (numbers). Or, are the numbers simply too sporadic and variable from day-to-day/situation-to-situation?

My proximity: North America, Mid-West, near/fairly about the Canadian-American border.

Thanks man.

18

Is it safe to ask about Social Matchmaking in here without having my post removed or the thread locked?

Josh - Is there likely to be any changes to social matching to prevent rubbish tier players like myself solo queueing and being matched against pre made teams of 4 featuring Champ/Onyx level players? I play social in order to enjoy the small of amount of time I get to play Halo in a non sweaty capacity, the last thing I want is to get pub stomped by teams of players that have reached levels I could only dream of reaching. Surely even in social playlists there should be some system in place to prevent these kind of things happening?

I get that it’s social and that I could go into ranked to play folks at my skill level but surely that defeats the point - why even have a social playlist if the answer is ‘play ranked’? Ranked play even at my level can be sweat fests (made worse by quitters and the subsequent no JIP, no mics and of course matching pre made parties). I have always (since H3) seen social as the place to play and enjoy the Halo experience, not get matched against god tier talent that wants to inflate their KD ratio.

19

> 2535466676504228;18:
> Is it safe to ask about Social Matchmaking in here without having my post removed or the thread locked?
>
> Josh - Is there likely to be any changes to social matching to prevent rubbish tier players like myself solo queueing and being matched against pre made teams of 4 featuring Champ/Onyx level players? I play social in order to enjoy the small of amount of time I get to play Halo in a non sweaty capacity, the last thing I want is to get pub stomped by teams of players that have reached levels I could only dream of reaching. Surely even in social playlists there should be some system in place to prevent these kind of things happening?
>
> I get that it’s social and that I could go into ranked to play folks at my skill level but surely that defeats the point - why even have a social playlist if the answer is ‘play ranked’? Ranked play even at my level can be sweat fests (made worse by quitters and the subsequent no JIP, no mics and of course matching pre made parties). I have always (since H3) seen social as the place to play and enjoy the Halo experience, not get matched against god tier talent that wants to inflate their KD ratio.

Yeah, Social is always a bit schizo. Good players want to “relax” meaning be able to own without much work. Bad players want to be “safe”, meaning to not get owned and be able to play.

Right now, it’s a sort of compromise between the too. Though I’m considering making it randomly choose between much tighter and much looser than now, so you get some safe and some crazy.

Have you tried searching focused? If you search Focused, it till try and give you tighter matches for about 40 seconds in Social before it switches to “anything goes” mode.

Looking at the data, I would expect the least imbalance in:

  • Ranked SWAT - Super Fiesta - Ranked SlayerIn that order. So, yeah, I know it’s weird that you would have to play “Ranked” for a safe experience, but that’s the current state of things.
20

> 2533274901013958;17:
> Excellent updates, as always, Dr. Menke. I always enjoy reading your thoughtfully composed, accurately written, and transparent updates.
>
> Is there an average wait-time differential 343 has been able to garner from their data from the MM servers, in comparing “Focused” matchmaking vs “Expanded” selection? I traditionally have always chosen balance, so I’d just like an answer from 343 regarding these time figures (numbers). Or, are the numbers simply too sporadic and variable from day-to-day/situation-to-situation?
>
> My proximity: North America, Mid-West, near/fairly about the Canadian-American border.
>
> Thanks man.

On average Balanced and Expanded will match faster than Focused, though won’t be as tight and potentially worse ping.