Using wins or win / loss tends to work in live leagues and sports for reasons that don’t apply online. Including fixed teams, fixed expectations about teams, a fixed number of matches, a fixed schedule, skill-based bracketing being pre-determined by a rigorous, selection process, and the players either paying or being paid to show up.
You say, “any pro sport proves this” which ironically completely disproves your whole suggestion because being a “pro” requires you to qualify, which is the same as skill-based matchmaking. You can’t even play in matches (e.g. matchmake) against pros unless you’ve proved you’re a pro. That is where your skill-matching happens. Players are put into major, minor, farm, and city leagues in “real” sports based on their skill, and are promoted / demoted as appropriate. So, yes, “pro sports” prove you need matchmaking.
Online throws all the things that make win-based systems work out the window, and make it so your suggestion just won’t work. Teams are the opposite of fixed, even matches and even teammates are expected (which using win-based matchmaking categorically will not give you), players can play as many matches as they want, players often don’t show up, and no one is paying / getting paid to play a specific set fixed number of games. You can play any time. If you DO try and use fixed times, players again, don’t show up (happened all the time in tournament systems I’ve worked on).
You can’t use number of wins. It’s just a bad skill measure. Take pro sports again. Who cares how many wins a pro team has if 20% of them were against 8-year olds (that’s the kind of skill gap we deal with online). And number of wins is wrong all the time when you consider some mediocre players just play a lot more games than some “Pro” level players, and will have more wins than them.
Even more importantly, # of wins is bad at predicting who will win between two teams, which means it’s not predictive of who is a good team / player, since being able to win is the most important skill to have. If you matchmake players using a system that’s bad at measuring player skill (like win count), quit rates skyrocket if you have a competitive game. Like, why would any 8-year old little league player want to play a full game against major league players? It’d just be dumb.
We see overwhelming evidence that if the other team is better than you are, you have a sky-rocketing chance of just quitting out early. This leads to everyone have a bad experience. Unfortunately, win count won’t catch that. You need something like TrueSkill. It’s even worse if one team is better than the entire other team. Can’t even play if everyone is quitting. Even if they don’t quitting out of a given match, they stop playing after that and don’t come back. And then you don’t have any kind of competitive experience because no one is playing — I’ve seen this in several older games before solid matchmaking was developed.
So you need a good measure of skill to matchmake well.
You should also use a good measure of skill to rank your players, otherwise no one will care about the ranks.
Ranking based on number of wins in a system that allows everyone to play as much as they want (unlike live events) is the same as ranking based on # of games played, and quickly becomes meaningless and a measure of who has more time to play, instead of a measure of actual skill.
Again, you need your ranking system to be predictable of who will actually WIN, since otherwise, that’s not skill, just participation trophies.
Also again, live event sports have a fixed number of games to play, so win count is the same as win%. Even win% itself is bad unless you play a correct sample of opponents at the correct skill-level. In major league sports, the teams all player each other, sometimes many many times, so win% is a great measure of skill.
But online, you can go a whole season without seeing the same team twice, so you need something smarter than just wins or even win%. Also, online, the opponent difficulty can be all over the place, subject to the whims of the current population. Which is why you need something smarter like TrueSkill to adjust for schedule difficulty.
So, yeah, there are important reasons why what works for live sports doesn’t work online.
