Priors for Unrated Players
When Ratings Central processes an event, it first assigns a prior mean and standard deviation to each unrated player in the event. The adjective “prior” means that it is the player’s mean and standard deviation prior to the start of the event. If you specified a prior mean and standard deviation for the player, then it uses those values. If you didn’t specify a prior mean and standard deviation for the player, then it uses the prior mean and standard deviation that you specified for the event.
The following sections explain how to set the individual player prior means and standard deviations and the event prior mean and standard deviation. This will give you an idea of how the system works. However, it isn’t easy to set good priors.
You must get our approval for the approach that you will use to set priors! To get our approval, .
Please get unrated players at your event as many matches with rated players of a similar playing level as you can. If you cannot get each unrated player several such matches at your events, please for assistance.
The Fewer Priors the Better
Priors are necessary, but the fewer priors, the better. Every time that you set a new player prior or event prior, you are adding new parameters to the rating system model. Each new parameter may be wrong or may have unintended consequences. The best approach is to determine a single prior that describes your entire population of players, then use that single prior as the event prior for all your events and not use player priors. If there isn’t sufficient mixing (i.e., players of different levels playing each other), this may not work, in which case you will have to use different priors for subpopulations.
If you are just starting to use Ratings Central, and so most of your players are unrated, you may find that if you use one prior for everything, it takes a few events for the ratings to become accurate. It is generally best to live with such a “burn-in” period rather than try to use priors to make every player have an accurate rating immediately. Letting the ratings be determined by the match results rather than by the priors is also fairer. Backloading historical data is a good way to get things started. Such data can also be used to evaluate the effect of different priors. We can assist with such investigations.
The following sections explain how to set player priors and event priors. But, to reiterate, it is best to not use player priors and to use the same event prior for all your events. If you feel that this won’t work for your situation or if you have already submitted events and it isn’t working, then for assistance.
The following advice for setting player priors assumes that you are very familiar with the rating scale, i.e., you know and play with many players who have established ratings. If this is not true, then you should rely on the event prior and rarely set individual player priors. Even if you are very familiar with the rating scale, evaluating individual players takes effort to do well, so relying on the event prior for some or all players can be simpler, more reliable, and fairer. If you aren’t sure what to do, please for assistance.
Despite the cautions in the previous paragraph, it can be helpful to set the player prior mean and standard deviation for an unrated player. However, you should only set the prior mean and standard deviation for an unrated player if you have additional information about that player. The “additional information” can be any information other than the player’s match wins and losses in the event (Ratings Central sees those). For example, you might know the player from before the event or you might watch the player play their matches or you might look at how many points the player scored in their matches.
If a player is different from the general population of unrated players at the event, e.g., much better or much worse (perhaps because they are very young), then it is probably a good idea to set the player’s prior mean and standard deviation, rather than trying to stretch the event prior to include the player.
If many or most of the players in your event are unrated, and so the unrated players are playing mostly other unrated players, then it is probably a good idea to set the prior mean and standard deviation for some or many of the unrated players.
The prior standard deviation for a player measures how sure you are that you know that player’s playing strength. You should be willing to bet at 1:2 odds that the player’s playing strength is within one standard deviation of the mean, and you should be willing to bet at 2:1 odds that the player’s playing strength is more than one standard deviation from the mean. (Odds of 1:2 mean that you win $1 if you win the bet, but you lose $2 if you lose the bet. Odds of 2:1 mean that you win $2 if you win, but you lose $1 if you lose.)
For example, suppose you assign a prior mean of 1200 and a prior standard deviation of 100 to a player. Then you should be willing to bet at 1:2 odds that the player is really between 1100 and 1300, and you should be willing to bet at 2:1 odds that the player is really less than 1100 or more than 1300. Equivalently, you should believe that there is a 2⁄3 chance that the player is really between 1100 and 1300 and a 1⁄3 chance that the player is really less than 1100 or more than 1300.
Here are some very rough guidelines: If you know an unrated player extremely well (e.g., they’ve been playing at your club every week for a couple of years), then you might use a prior standard deviation of 100. If you only know a player moderately well (e.g., they came to your club a few times and played several matches with players of a similar level), then you might use a prior standard deviation of 125. If you know very little about a player (e.g., you had the player hit with a rated player of a similar level for five minutes), then you might use a prior standard deviation of 175.
Be cautious using low values for prior means or for prior standard deviations, as these can pull the ratings of other players down or make it difficult for a player’s rating to change. It is harder to estimate the level of players below 1000 because these players are less consistent. So, you should use larger standard deviations for such players than you would otherwise. But, see Rating System Floor below.
If a player has a USATT rating, then that is a source of information. However, for the reasons given in Rating Scale, it is not possible to simply convert USATT ratings to Ratings Central ratings. You will have to use your judgment in how you let a player’s USATT rating influence the prior mean and standard deviation that you use. It is very unlikely that you will want to use the player’s USATT rating as the prior mean. Do not make the mistake of using too small a standard deviation when relying on a player’s USATT rating.
You must always set the event prior mean and standard deviation.
While the prior mean and standard deviation for a player measure what you know of the player’s playing strength, it is best to interpret the event prior mean and standard deviation as describing the range of unrated players at your event. For example, if you think the unrated players range from 800 to 1400, then you would use the average of these two values (i.e., 1100) as the mean and the difference of these two values divided by four (i.e., 150) as the standard deviation.
More precisely, about 2⁄3 of the unrated players should be within one standard deviation of the mean (and about 1⁄3 should be more than one standard deviation from the mean), 95% should be within two standard deviations, and 99.7% should be within three standard deviations. So, for the example in the previous paragraph of a mean of 1100 and a standard deviation of 150, you should think that
- 2⁄3 of the players are between 950 and 1250,
- 95% of the players are between 800 and 1400,
- 99.7% of the players are between 650 and 1550.
Note that when estimating the event prior standard deviation from the range of players that you expect at your event, you should interpret the range as being plus or minus two standard deviations, not three.
If you set the prior mean and standard deviation for any individual unrated players, then the event prior mean and standard deviation should only describe the population of unrated players for whom you haven’t set individual prior means and standard deviations. If you set the prior mean and standard deviation individually for every unrated player at your event, then Ratings Central won’t actually use the event prior mean and standard deviation, but you still have to set them.
It is possible that a player that you thought was rated may become unrated because of a correction to some other event. If this happens, the event prior mean and standard deviation would be used for the player, even though you thought it wouldn’t be. But, this eventuality is extremely unlikely, so not worth worrying about.
Rating System Floor
While the system allows ratings to range from 1 to 3500, we don’t expect an adult player to be less than 500. Do not set a player prior or event prior such that the mean minus twice the standard deviation is less than 500 unless you have first and we have approved it. This applies to both adults and children.
If you want to find what unrated prior was used for a player and what their first event was, you can look at the player’s rating history page, e.g., Rating History for Sean O’Neill. You can also get this information by using the FirstEvent.php page. This page takes most of the same parameters as the PlayerList.php page. The differences are that you can’t specify the as-of date and the output is unsorted CSV with columns PlayerID, ReportID, InitialMean, InitialStDev, PointChange, StDevChange, EventDate, EventDirector. Note that the PlayerSport parameter is required; it can’t be omitted or be an empty string. Perform a search using the Players page to see the parameters. Also, the PlayerID parameter can be a list, like the Players (multiple IDs) page does. The FirstEvent.php page accepts both GET and POST requests.