This is a pretty long post about properly using data to inform design decisions. From what I can tell, the metrics being used to determine the ‘competitiveness’ of maps and quality of weapon balances aren’t being decided properly. I think this is why we’ve landed on the poorly received weapons tunings, and why overgrowth is considered a competitive map.
How 343 measures maps:
343 believes that a map is ‘competitive’ if (1) Red and Blue have equal win rates and (2) higher skilled teams win more often than lower skilled ones. Neither of these have anything to do with how competitive a map is.
Their first metric actually measures fairness. Having equal win rates just means neither side has inherent advantage. Competitive maps should be fair. But fair maps aren’t neccisarily competitive. For perspective: A Coin flipping contest would be competitive by this metric.
Their second metric actually measures the influence of skilled elements. Higher skilled teams should always have an advantage because they are better at the game. Seeing that this advantage is typically converting into wins isn’t a sign of a highly competitive map. It’s just a sign that skill can usually overcome whatever non-competitive issues are present.
How they should be measuring maps:
They should be less interested in whether or not the expected team wins, and more interested in what skills need to be displayed in order to win.
If grenade proficiency is flat, regardless of skill level, then the maps is probably rewarding of grenade spam instead of skillful placement.
If grenade and Melee damage dominate the damage dealt, then those lesser skilled elements are overshadowing aim skill.
If many engagements aren’t ending in death, then there might be site line issues.
They need to examine heat maps to ensure that there aren’t impregnable positions or overly dominate strats.
They need to look at survival rates off spawn to see if spawns are too predictable for a team to recover.
Basically, there are a million different things that they need to measure to see if their maps are competitive. And if their maps were truly competitive, they wouldn’t even have to check the measurements they’ve been testing because those would automatically work themselves out.
They’ve got similar issues when gauging the sandbox. For example the BR:
Menke mentioned that the current BR isn’t less competitive than its previous version because 1) is has similar spread to the BR in previous Halos, and those games were considered competitive and 2) better players are win the majority of their engagements with lesser skilled players.
The first one point is nonsense because in previous halo’s extreme measures were taken by TOs to mitigate the RNG Bungie added to precision weapons.
The second point runs into the same issue as their map metrics. the question isn’t whether or not skilled players will usually win. It’s whether or not skillful execution is THE determining factor. skill can help mitigate RNG, but it can’t eliminate its influence. Therefore the outcome between similarly skilled players has a high change of being decided by luck rather than execution- and the outcomes would still be 50/50, which would through of the interpretation of the data.
And regarding the magnum:
It’s been said that the magnum isn’t fit for competitive because, on paper, it now outclasses all the other rifles- as seen on the TTK and Range charts they published.
The problem is, the charts only detail how effectively these weapons CAN be used, not how effectively they ARE used in practice.
Their comparisons need to include, means and medians for killtimes and engagement ranges, in order to gauges how players are ACTUALLY using these weapons and to understand how skill effects usage. It would also be useful to break this down by skill divisions. Since the magnum isn’t included in the test, not only is the claim invalid, but they aren’t gathering the info needed to get the sandbox properly balanced around the magnums inclusion.
I know a lot of people are totally against analytical development ,but there’s literally nothing wrong with the concept. The problem arises when the incorrect data is used or data is used incorrectly, which is what i believe happening here.
Thoughts?
Please don’t cross post topics, thanks
https://www.halowaypoint.com/en-us/forums/58b8518e005f432381ab99fbcaf931e0/topics/halo-5-designing-by-data/5e33fce9-aabc-4d44-a336-426a82879f62/posts?page=1