ratings cheat

[Chuck Kennedy]

My only gripe is that some rounds get dropped from peoples rounds if greater than some statistical variance.

We only drop rounds more than 2.5SD below a player's average or more than 100 points. That amounts to one round out of 50 normal rounds on average. Players can shoot an exceptionally poor round by choice but not an exceptionally great round so this process is necessary to prevent players from easily pulling down their rating with poor rounds.

[Flash]

Originally Posted by Chuck Kennedy

We only drop rounds more than 2.5SD below a player's average or more than 100 points. That amounts to one round out of 50 normal rounds on average. Players can shoot an exceptionally poor round by choice but not an exceptionally great round so this process is necessary to prevent players from easily pulling down their rating with poor rounds.

Yes but a round that is greater than 2.5 SD above their rating artificially inflates a rating especially if it is double weighted. Now this may not be a factor for people with a large number of rounds but it is for people with a small number of rounds, like say 10 rounds, if one of the last 3 rounds was a smoking hot round it will skew their true player rating.

Have you ever thought of including all rounds but not double weighting them if the round is outside 2.5 SD either side of their rating?

[Chuck Kennedy]

By definition, any round even 3 standard deviations above a player's average is still in their "normal" range. It's only rounds on the low side that are potentially subject to manipulation, injury or abnormal penalties such as a player showing up late and getting par+4 for several holes. If a person is improving, then that very high round rating may be more typical of their current performance so doubling it is not out of line. No rounds are permanently doubled. That very good round will not be doubled after 3-4 months and a player will have to continue with higher rounds to maintain that improvement.

[Flash]

Originally Posted by Chuck Kennedy

By definition, any round even 3 standard deviations above a player's average is still in their "normal" range. It's only rounds on the low side that are potentially subject to manipulation, injury or abnormal penalties such as a player showing up late and getting par+4 for several holes. If a person is improving, then that very high round rating may be more typical of their current performance so doubling it is not out of line. No rounds are permanently doubled. That very good round will not be doubled after 3-4 months and a player will have to continue with higher rounds to maintain that improvement.

Chuck if this was some kind of process that had an upper and lower limit then 3 SD might make sense but the only limit is the one you impose by assigning a combing effect at 2.5 SD. Sure people have more control over their rating at the lower end but that doesn't mean you skew the data to prevent that. You need to take care of that in other ways. If someone is shooting a round in 10 rounds that is off the SD charts for them in the upper direction why double weight it even for the 3-4 month period. All you are doing is causing a ripple effect in their rating if they can not sustain the hot round every ten rounds and if they do continue it would be included in the ratings but just not at a rate of double the effect until that level of play was within their 2.5 SD mark!

In my experience, people don't tank a round to lower their rating, more often they tank a round to get below the 2.5 SD you impose so that it will not be included in their update. If people were able to tank a round and lower their player rating in hopes to get below some cutoff in the amateur ranks so they can sandbag that division then the next few rounds will more then likely push them above the cutoff again as they try to win another event in Int.

I guess I can't say that I am really trying to change your mind on this, just trying to let you know how someone outside of the ratings systems perceives your method

[Chuck Kennedy]

Remember that the fundamental rationale for the ratings system was to provide a way to get amateurs into the proper skill divisions and prevent sandbagging. All tweaks have been made with that being the highest priority. The companion to this priority is to do our best so that every PDGA member gets ratings. Ratings accuracy takes a backseat to those priorities. Dropping rounds below 2.5SD makes it more difficult (and expensive) for players to pull down their rating (anti-sandbagging). Double weighting attempts to offset the lag in ratings to better reflect how fast improving ams are currently playing (anti-sandbagging). Not doing ratings updates more often than 6 weeks reduces volatility, encourages more pre-reg for events making less work for TDs moving players to different divisions. Doing round ratings with as few as 5 propagators makes sure most if not all rounds get rated so members get ratings.

Everyone of those choices degrades ratings accuracy a little bit. But the bigger goals of the org are met and the system is functional for those reasons. It's practical versus pure stats.

[REDFIVE]

With this being the case would it be beneficial to change the way pro ratings are calculated to become more accurate?

[Chuck Kennedy]

There's no reason for more precision for pros that offsets the value of doing them the same way for everyone and making sure that everyone gets a rating each round. There are several pro divisions like women that sometimes play different layouts or in different pools that might not get ratings if some of the procedures were changed. The ratings used for World Rankings are theoretically more accurate for historical comparisons because rounds are limited to B-tiers and higher, there's no double weighting and everyone has rounds only in the same 12-month period.

We are currently looking at statistical ways to improve the calculations when possible but I doubt it will be noticeable. Regardless how accurate the ratings become, they are only an average around a distribution of possible future round scores. They can't predict precisely what a player will shoot, just a range. That's a good thing, otherwise we could just send in our entry fees to an event without playing them and the TD would just send back the payouts based on the ratings of those who entered.

[Flash]

Originally Posted by Chuck Kennedy

We are currently looking at statistical ways to improve the calculations when possible but I doubt it will be noticeable. Regardless how accurate the ratings become, they are only an average around a distribution of possible future round scores. They can't predict precisely what a player will shoot, just a range. That's a good thing, otherwise we could just send in our entry fees to an event without playing them and the TD would just send back the payouts based on the ratings of those who entered.

Careful Chuck that may depend on whether or not the rating is a Homey rating and if the event is not at a homey course

I know my opinion on this is not a popular one but I don't believe Pro's should have PDGA ratings, they should simply have a world ranking. The ranking will depend on various factors but a rating system that supports the proper placement of amateur players would not be one of them. Basically there are no reasons pro need ratings from a PDGA standpoint and a world ranking system would be better geared towards sponsorship deals and avoid homey ratings to some extent. There would need to be some more work around this to hammer it out but the world rankings are doing a pretty decent job of evaluating Pro capabilities in a variety of playing conditions and challenges in different areas of the world. I am sort of shocked that the rating system is geared more toward supporting the Organization instead of the member, it has always been sold as a feature to the member!

[Chuck Kennedy]

Roger and I did the ratings free as volunteers from 1998 for six years before the PDGA started to provide some compensation. Our goal was that ratings be used for divisional breaks for ams or we would stop doing them which we did in 2001 to protest. Theo Pozzy who was Board member basically guaranteed the ratings system would continue by developing divisional guidelines with us and ratings events by 2002. While Roger and I plus many others wanted the ratings for divisional guidelines, the overwhelming number of members just thought having ratings was cool. So that became a side benefit which forced some of the decisions indicated above so that all members get ratings most of the time.

As head of the Course Committee and active designer, the alternative reason I got excited about developing ratings from the very beginning was it provided a way to produce course ratings (SSA). I really never cared too much about personal ratings but getting an objective way to evaluate course scoring was and continues to be a key motivation.

Ratings are still fundamental for World Rankings (40-70% of the ranking points) since we still only have few events globally where the top players face each other. If we get to the point where there are 15-20 events of the caliber of the top NTs and better, then we'll be able to gradually phase out the weight for ratings and use head-to-head performance more often.

[REDFIVE]

I enjoy having and following ratings so I thank you for doing what you do.