Arena System Update for Jan. 23rd

Here’s the breakdown of all the arena issues going on right now. As far as I know (since the servers are down) the arenas are still shut down. I wish I could write more but the blizzard quotes pretty much sum everything up. The most interesting bit about this entire thing is that Blizzard wants to rollback all ratings to how they were before the patch hit.

Earlier this week the Arenas were temporarily disabled while we investigated rating calculation discrepancies. After thorough research, we have identified and created a solution to the issue, and we will be bringing the Arenas back online following off hours maintenance with the fix in place. Please be aware that because the rating issues were extensive, we have decided to reset Arena team and personal ratings to last week’s status. In addition, items and achievements which were gained illegitimately with incorrect ratings will be removed, and the spent arena points refunded. We thank you for your patience while we worked through these issues related to the Arenas. (source)

Kalgan gave a lengthier response with more detail however, I don’t know how much good it’s doing but it’s worth the read if you care about arenas in any way and desire to know what the hell just happened the past two days. (source)

- The team/personal ratings and the hidden ratings are related to each other. Your team/personal ratings essentially drift toward your hidden rating, so if you deliberately lower your hidden rating and remake a team, your team/personal ratings simply won’t get up to high ratings until your hidden rating does. His exploit as proposed simply doesn’t work because his assumptions about the relationship between the visible and hidden rating are incorrect.

- Yes, a group of good players could still deliberately stomp nubs by intentionally losing lots of games to lower their hidden rating. A rating system won’t stop that, but it still solves most cases of high rated players beating up on lower rated players. First, many of the cases where high rated players were being matched against low rated players wasn’t because the high rated players were deliberately trying to stomp people, it was often accidental as high rated players had normal/natural reasons to change teams on occasion (helping friends, trying out new players, new comps, new strategies, etc). Second, it takes far more effort to deliberately deflate your rating down to low levels than it did to simply create a new team.

- Something that many of you have mentioned is that the rating adjustments are now not always “zero sum”. However, a rating system doesn’t need to be zero sum in order to prevent inflation. The new system uses a bayesian prior distribution in order to prevent inflation, which in simpler terms means there’s math that enforces that players won’t have ratings outside of some range (when working without bugs, of course). For the population size we have for arenas, this means we should see ratings range from around 600 to 2400 (again, assuming no bugs).

- There were two bugs with the system that we’re in the process of fixing before we reactivate arenas (should be soon) that were causing serious problems (and distorting everyone’s perspective on the system). One was in the functionality that seeded everyone’s hidden rating based on their pre-3.0.8 patch performance, causing players to “drift” toward wildly inaccurate ratings, the other being in the way that personal ratings were adjusted.

Effects of rating changes in the middle of the season
I think you say that with the assumption that if it had been the start of the season, the bugs wouldn’t have been there (not a good assumption in this case imo). The new system (when working) doesn’t significantly change how hard it is to get the gear or who you have to beat to get the title/mount/etc (that’s still entirely relative). It also tends to be mid/late in the season when the problems with the old system were the most exposed (with highly rated players starting new teams, helping friends, etc).

“Hidden” rating
It is not like personal rating. The hidden rating uses a different approach to determining player ratings. It’s more complex, but it tends to figure out a player’s skill level far more quickly/accurately than an Elo system does.

However, changing player’s actual team/personal ratings extremely rapidly isn’t exactly what we want. So, we’re using an approach where we use the hidden rating (internally we refer to it as the GDF rating since it uses gaussian density filtering) for matchmaking since it creates better matchups than Elo and we use it as an anchor to figure out where the team/personal ratings should be moving toward.

Rollback
Yes, I’m implying that we hope to roll back team/personal ratings back to what they were right before 3.0.8 went live.