Wrong Ratings

Ratings should accurately reflect the playing strength of the player to within the stated accuracy (i.e., to within two, or rarely three, standard deviations). There can be several reasons why this might not be true.

Sometimes a player’s results simply do not reflect their true playing strength. The ratings can only tell us how strong a player was when they played in submitted events, not how strong they were or are at other times. When deciding if a player’s rating is wrong, do not consider information that is not available to (or not used by) the rating system.

If that is not the problem, then the most likely reason a rating might be wrong is that the priors that were used are not good, either for the player with the wrong rating or for other players in the population. See Priors for Unrated Players for how priors should be set.

An unlikely, but possible, reason is that matches are being submitted in too many individual events rather than combining them into fewer events. See Effect of League Grouping for more information.

Rarely, a player may do something so unusual that the rating system finds it hard to believe, e.g., a player might actually increase their playing strength by 400 points in one month. In such a case, the rating should catch up to the player’s playing strength as more results are submitted. This is not a flaw in the system: The system must make some assumptions about what is likely in order to make sense of the data.

When looking at ratings, it is important to look at the rating that the player ends up with and not be too concerned with the way that the system comes up with this value, i.e., do not be too concerned with the number of points gained or lost for an individual match. These match point-change values are not well-defined and depend on many things. See How the Rating System Works and Examples of Point Changes for more information.