Page 1 of 5
Permaximum Ratings and Rankings (Updated Often)
Posted: Wed Dec 26, 2012 11:18 pm
by permaximum
Since there is no pure empirical box-score ratings for players (probably because of assist weights) I wanted to make one and called it PTR(Player Total Rating). Used 1997/1998 - 2011/2012 data. I chose 97/98 as the starting point because of 3-point line rule change in 97 summer. I'm especially curious about how PTR will compare to PER when it comes to player effect on team success.
Also, since there is no complete team ranking system, I made one which takes MOV, SOS, HCA, B2B and PACE into account. It's a team-level ranking system as you probably guessed so it doesn't care about injuries.
First; PTR of top 30 players in NBA.
PLAYER TOTAL RATINGS
Code: Select all
Player PTR
1. LeBron James 30.63
2. Kevin Durant 29.92
3. Tim Duncan 29.21
4. Andre Drummond 28.41
5. Tyson Chandler 28.20
6. JaVale McGee 27.81
7. Andray Blatche 27.09
8. Carmelo Anthony 26.94
9. Blake Griffin 26.74
10.Anderson Varejao 26.73
11.J.J. Hickson 26.34
12.Robin Lopez 25.79
13.Kobe Bryant 25.33
14.Chris Bosh 25.12
15.Larry Sanders 25.07
16.Serge Ibaka 24.97
17.Chris Paul 24.83
18.Ed Davis 24.83
19.Kenneth Faried 24.77
20.Zach Randolph 24.49
21.Tiago Splitter 24.32
22.Dwight Howard 24.23
23.Al Jefferson 23.80
24.David Lee 23.61
25.Ryan Anderson 23.26
26.Kosta Koufos 23.10
27.Amir Johnson 22.94
28.Derrick Favors 22.81
29.Paul Millsap 22.72
30.James Harden 22.55
Box-Score stats do not represent players' defensive effect other than block and steal (e.g contesting shots, indirectly causing turnovers etc.), superstars' ability to draw double-teams and make contested shots, guards' ability to bring the ball, run plays for the team and create chances. Because of these reasons, you won't really see guards and bruce bowen, ron artest type of defenders at top. Notice Chris Paul at 17 and Kobe at 13. In the same time PTR doesn't like assists as much as PER does too. I have data for past years and I can confirm that there hasn't been any point guard or small players in top 5 except Chris Paul and Dwyane Wade. Only Allen Iverson got into top 10 in a season besides those two although PTR penalises more for misses than PER. IMO it's more about the fact that Iverson was a high-efficiency player in reality (although he has 43% career fg) when it comes to +30% usage players.
Second; team-total PTR of NBA teams and their PTR positions' comparison to MOV positions.
TEAM-TOTAL PTR AND MOV COMPARISON
Code: Select all
Team PTR PTR-MOV P
1. LAC 95.92 0
2. OKC 94.53 0
3. MIA 91.74 -1
4. NYK 90.62 -1
5. SAS 89.79 +2
6. DEN 86.07 -6
7. LAL 84.85 0
8. HOU 84.57 0
9. UTA 84.02 -8
10.MEM 82.43 +4
11.GSW 81.05 0
12.ATL 80.70 +3
13.BRK 80.66 -2
14.PHO 80.02 -10
15.BOS 79.92 -1
16.MIN 78.89 +3
17.PHI 78.46 -5
18.POR 77.95 -3
19.MIL 77.29 +1
20.CHI 76.88 +6
21.DAL 75.91 -2
22.CHA 75.59 -8
23.TOR 75.23 -2
24.SAC 74.56 -2
25.NOH 74.06 -3
26.IND 73.98 +16
27.DET 73.74 +7
28.ORL 71.87 +9
29.CLE 68.82 +2
30.WAS 64.30 +1
PTR fails at explaining the MOV positions of 10 teams. DEN, IND, ORL, DET, CHA, UTA, PHO, PHI, MEM and CHI (Misplaced by more than 3). It looks they are either very good or very bad defensive teams compared to their PTR position. Especially Pacers must have top-class bruce bowen, ron artest type defenders. I wonder how PTR compares to PER in this regard. Bear in mind PTR adjust for 100 poss while PER adjust for the league average poss (91.85 atm).
TEAM-TOTAL PER AND MOV COMPARISON
Code: Select all
TEAM PER PER-MOV P
1. LAC 86.57 0
2. OKC 86.56 0
3. MIA 84.96 -1
4. NYK 84.25 -1
5. SAS 82.92 +2
6. DEN 79.90 -6
7. LAL 79.28 0
8. HOU 78.88 0
9. UTA 77.60 -8
10.MEM 76.36 +4
11.BRK 75.72 -4
12.GSW 75.65 +1
13.ATL 75.39 +4
14.PHO 74.92 -10
15.BOS 74.23 -1
16.POR 73.66 -5
17.PHI 73.63 -5
18.MIN 73.30 +5
19.MIL 72.32 +1
20.CHI 71.88 +6
21.DAL 71.65 -2
22.CHA 71.34 -8
23.SAC 70.97 -3
24.DET 70.24 +4
25.NOH 70.04 -3
26.TOR 70.00 +1
27.IND 69.49 +17
28.ORL 68.31 +9
29.CLE 65.84 +2
30.WAS 61.13 +1
It looks very similar to PTR. PER fails at explaining the positions of 14 teams (misplaced by more than 3). Overall PER performs 7.55% worse than PTR. No surprise for Pacers. Misplaced by 17 positions. I thought the big difference in assist weights between two metrics would give considerably different results but it seems it's nearly evened out by the difference in other weights.
Last, team rankings. MOV adjusted by SOS, HCA, B2B and Pace. HCA is different for each team. Pace adjustment made for 100 possessions. I think this can be the best and most accurate metric for quickly analyzing team performances in NBA. NoPaceA is the rank without pace adjustment.
TEAM COMPLETE RANKINGS
Code: Select all
Team C.Rank NoPaceA
1. LAC 10.72 9.85
2. OKC 9.23 8.56
3. SAS 8.44 8.00
4. NYK 5.90 5.38
5. MIA 5.52 5.09
6. MEM 5.45 4.92
7. ATL 3.51 3.22
8. LAL 2.59 2.44
9. DEN 2.58 2.42
10.CHI 2.08 1.87
11.HOU 2.05 1.99
12.GSW 2.03 1.90
13.IND 1.54 1.39
14.BRK 1.48 1.30
15.MIN 1.33 1.23
16.BOS 0.06 0.06
17.MIL -0.95 -0.89
18.DET -1.18 -1.06
19.UTA -1.74 -1.59
20.PHI -3.13 -2.82
21.POR -3.20 -2.92
22.ORL -3.76 -3.46
23.DAL -4.23 -3.99
24.TOR -4.54 -4.11
25.PHO -4.63 -4.28
26.CLE -5.10 -4.67
27.NOH -5.92 -5.20
28.SAC -7.19 -6.63
29.CHA -9.16 -8.45
30.WAS -10.48 -9.56
I think 12-13 NBA champions will be one of the top 5 teams. Clippers, Thunder, Spurs, Heat and Knicks. Memphis would be a big surprise. Lakers... very little chance.
Re: Permaximum Ratings and Rankings (Updated Often)
Posted: Thu Dec 27, 2012 11:22 am
by Mike G
The team rankings are unchanged whether or not you adjust for pace.
It's often been noted that PER for players do not yield an accurate summary for the teams: offense is rewarded, and defense is not.
I get a pretty large weight on assists by a few adjustments:
-- adjusting players' scoring by their (estimated) % of points which are unassisted; this gives more credit to players who create shots (like point guards) and less to players who are assist-dependent.
-- adjusting to home/away disparity. Basically, a team or player's away assist rate is his real assist rate.
-- adjusting to the team ratio of Pts/FG. A 3FG is 1.5 times the contribution as a 2FG, so an assisted 3FG should be worth more. A lot of team FT also suppresses Assists, so that may get bumped up.
Also I get better correlation between teams and their players' rates when adjustment is made for starters and bench players. A formula estimates the % of opponents who are starters, and this gives the biggest boost to starters who play modest minutes; discounts bench guys who play few minutes.
This factor applies to all production stats equally. PerMax has Javale McGee, Andre Drummond, and Andray Blatche among the top 7 players.
Blatche has started about 1/4 of games and goes 21 mpg. I estimate just 56% of his opponents have been starters, and his totals are factored by .96
The others have not started and avg 19 mpg. At about 46% vs starters, I factor their rates by .90 .
Without such a starter/sub adjustment, you have to use an arbitrary minutes cutoff when you list 'top players'. And you still get not-ready-for-primetime players sneaking in.
Carry on.
Re: Permaximum Ratings and Rankings (Updated Often)
Posted: Thu Dec 27, 2012 1:40 pm
by permaximum
Actually I agree with all the things you said. In fact, at one point I was going to make a more reliable metric which would contain "estimated" (more error) advanced stats like, assisted fg etc.
But in the end, I thought that you or someone else who does this work more seriously, must already have a good metric for that. So that I decided to only go with "point, offensive rebound, defensive rebound, assist, block, steal, turnover, personal foul, 2p miss, 3p miss, ft miss", simply pure box-score stats. I know those other advanced stats could be estimated by using these stats, but I didn't want to make a complex metric with more error margin. If you go down that route, only RAPM will satisfy you.
Result, it's 7.55% better than PER so far and it's enough for me.
As for your claim about pace effect on team rankings, I want you to explain it to me.
Team A : +11 Pace: 90
Team B: +12 Pace 110
Which one is better? I fear I miss something here.
Re: Permaximum Ratings and Rankings (Updated Often)
Posted: Thu Dec 27, 2012 4:30 pm
by Mike G
As for your claim about pace effect on team rankings, I want you to explain it to me.
In the last list in your opening post, teams with and without pace adjustment, the teams are ranked in the same order, top to bottom. There are just a couple which come close to being different, but no.
Result, it's 7.55% better than PER so far and it's enough for me.
I think you're talking about players' minutes-weighted rates summing to team rates.
This could also be achieved by the simple metric (player pts/48)/(opponent team pts/48).
This would sum to (team pts)/(opponent pts), which is precisely equal to team strength.
However, you would be far off on fairly assigning credit to the individual members of a team. You have to give credit for rebounds, assists, etc. That's the tricky part.
Most people, whether they're statistically adept or numerically challenged, will not look at a metric very deeply, when they see subs from bad teams ranked among the top 10 NBA players, and well above known superstars.
Being "better than PER" in some esoteric correlation, while it says something, it is still dubious.
Re: Permaximum Ratings and Rankings (Updated Often)
Posted: Thu Dec 27, 2012 5:04 pm
by permaximum
In the last list in your opening post, teams with and without pace adjustment, the teams are ranked in the same order, top to bottom. There are just a couple which come close to being different, but no.
Actually Dallas and Philadelphia's positions are different with the pace adjustment. I thought you were pointing out a rule like "pace doesn't matter when it comes to team rankings".
I think you're talking about players' minutes-weighted rates summing to team rates.
This could also be achieved by the simple metric (player pts/48)/(opponent team pts/48).
This would sum to (team pts)/(opponent pts), which is precisely equal to team strength.
PTR= Value/min*48 + team pace adjustment.
I was talking about total player Value (in the formula above) of teams (total Value pace adjusted at last). Only to see how it performs. OFC (team pts)/(opponent pts) would be a lot better metric but the goal is not to calculate "team" strength but "player" strength by looking at simple box score stats. Players don't have exact opponents and their averages.
I mean it was a validation method. I am not trying to calculate team strength by using PER or PTR. How do you think a pure box-score metric like PER or PTR's accuracy should be tested then?
If you have a player-metric that takes advanced stats into account, I would test it "this way" again... Actually if you have one, I would like to test it this way to compare Estimation Error of Advanced Stats vs the Reliability of your formula. It would be good to see using advanced stats like assisted FG would worth the error of estimation.
Most people, whether they're statistically adept or numerically challenged, will not look at a metric very deeply, when they see subs from bad teams ranked among the top 10 NBA players, and well above known superstars.
Superstars to who? Check PER and you'll see a lot of subs at top too... This is what you'll get with per minute pure box-score metrics, I can't deny that. But that doesn't mean these type of metrics are invalid. Still you have to show me a proof that supposed superstar is actually better than that sub...
Edit: To make those people happy, I could base the metric on per-game instead of per minute stats and suddenly you would see those "superstars" at top. Then it makes me think coach decisions make superstars...
Re: Permaximum Ratings and Rankings (Updated Often)
Posted: Thu Dec 27, 2012 6:01 pm
by v-zero
Per-minute is fine, the best way to deal with the problem of low-minute subs is to treat the problem in a bayesian rather than frequentist way, and, if you like (though it's not a really big deal) do as Mike suggested and adjust player values by estimating percentage of time spent facing starter-level players rather than replacement-level. The combination of these two will, for the most part, keep your ratings reasonable with few obvious outliers.
Re: Permaximum Ratings and Rankings (Updated Often)
Posted: Thu Dec 27, 2012 6:04 pm
by Mike G
Check PER and you'll see a lot of subs at top too... This is what you'll get with per minute pure box-score metrics, I can't deny that. But that doesn't mean these type of metrics are invalid. Still you have to show me a proof that supposed superstar is actually better than that sub...
Right -- when you go with a pure per-minute stat, you get this kind of stuff.
A "proof" would be debatable at best. But you might poll head coaches, GM's, NBA players, or anyone else and see whether they'd pay more for Chris Paul or for Ed Davis.
I don't even know who Ed Davis is.
Re: Permaximum Ratings and Rankings (Updated Often)
Posted: Thu Dec 27, 2012 6:21 pm
by EvanZ
I use WARP just because it's easy and gives you the "total" productivity of players. (A Bayesian WARP would be even better.)
Is MVP given to the "best" player (e.g. best per possession) or the one who contributed the most total value to his team (e.g. by playing X number of minutes at a high level)?
Re: Permaximum Ratings and Rankings (Updated Often)
Posted: Thu Dec 27, 2012 7:18 pm
by DSMok1
v-zero wrote:Per-minute is fine, the best way to deal with the problem of low-minute subs is to treat the problem in a bayesian rather than frequentist way, and, if you like (though it's not a really big deal) do as Mike suggested and adjust player values by estimating percentage of time spent facing starter-level players rather than replacement-level. The combination of these two will, for the most part, keep your ratings reasonable with few obvious outliers.
Starter level defenders = replacement level defenders, for all intents and purposes. It's only on the offensive side that the differences arise.
Re: Permaximum Ratings and Rankings (Updated Often)
Posted: Thu Dec 27, 2012 10:04 pm
by Mike G
I am pretty sure that most players get more offensive rebounds when they're coming off the bench.
Re: Permaximum Ratings and Rankings (Updated Often)
Posted: Thu Dec 27, 2012 10:42 pm
by v-zero
DSMok1 wrote:Starter level defenders = replacement level defenders, for all intents and purposes. It's only on the offensive side that the differences arise.
Evidence? In any case it's still important to control for opposition strength in general.
Re: Permaximum Ratings and Rankings (Updated Often)
Posted: Thu Dec 27, 2012 11:40 pm
by DSMok1
v-zero wrote:DSMok1 wrote:Starter level defenders = replacement level defenders, for all intents and purposes. It's only on the offensive side that the differences arise.
Evidence? In any case it's still important to control for opposition strength in general.
http://godismyjudgeok.com/DStats/2012/n ... ent-level/
Re: Permaximum Ratings and Rankings (Updated Often)
Posted: Fri Dec 28, 2012 12:37 am
by Bobbofitos
Mike G wrote: Check PER and you'll see a lot of subs at top too... This is what you'll get with per minute pure box-score metrics, I can't deny that. But that doesn't mean these type of metrics are invalid. Still you have to show me a proof that supposed superstar is actually better than that sub...
Right -- when you go with a pure per-minute stat, you get this kind of stuff.
A "proof" would be debatable at best. But you might poll head coaches, GM's, NBA players, or anyone else and see whether they'd pay more for Chris Paul or for Ed Davis.
I don't even know who Ed Davis is.
That's an Ed Davis
Re: Permaximum Ratings and Rankings (Updated Often)
Posted: Fri Dec 28, 2012 1:34 am
by permaximum
v-zero wrote:Per-minute is fine, the best way to deal with the problem of low-minute subs is to treat the problem in a bayesian rather than frequentist way, and, if you like (though it's not a really big deal) do as Mike suggested and adjust player values by estimating percentage of time spent facing starter-level players rather than replacement-level. The combination of these two will, for the most part, keep your ratings reasonable with few obvious outliers.
Thanks for the suggestion but as I said before, this should be only the simple box-score stat metric like PER. Only difference is PER goes with theoritical approach while PTR goes with empirical approach. I don't want to get into the field of estimation because I would rather use RAPM if I can risk error margin. BTW while we're talking about RAPM do you know which cross validation method is used to find the lambda? I hope it's loocv but I doubt it since it would take "days" for a decent PC to calculate.
@Mike G
Ed Davis can be any NBA player who shoots 55%, grabs a lot of rebounds and turns the ball over 2 times or less per 48 minutes.
Anyway, Chris Paul is at number 13 now and passed our friend Ed. Meanwhile LeBron passed Durant and Kobe climbed up to 16. Here are the updated PTR for players and CRank (saying it again it's simply MOV adjusted by SOS, HCA, B2B and Pace) for teams.
Code: Select all
Player PTR
1. LeBron James 30.24
2. Kevin Durant 30.04
3. Tim Duncan 29.19
4. Andre Drummond 28.30
5. JaVale McGee 27.90
6. Tyson Chandler 27.87
7. Andray Blatche 27.79
8. Carmelo Anthony 26.92
9. Anderson Varejao 26.72
10.Blake Griffin 26.72
11.J.J. Hickson 26.53
12.Robin Lopez 26.33
13.Chris Paul 25.13
14.Ed Davis 25.06
15.Chris Bosh 24.98
16.Kobe Bryant 24.97
17.Serge Ibaka 24.85
18.Zach Randolph 24.77
19.Kenneth Faried 24.66
20.Tiago Splitter 24.50
21.Larry Sanders 24.44
22.Dwight Howard 23.90
23.David Lee 23.85
24.Derrick Favors 23.85
25.Kosta Koufos 23.60
26.Al Jefferson 23.59
27.Amir Johnson 23.43
28.Paul Millsap 23.00
29.James Harden 22.89
30.Ryan Anderson 22.89
Code: Select all
TEAM C.Rank NoPaceA
1. LAC 10.16 9.32
2. SAS 8.75 8.27
3. OKC 8.68 7.99
4. MIA 7.03 6.50
5. NYK 6.12 5.59
6. MEM 5.24 4.74
7. HOU 3.42 3.29
8. ATL 2.94 2.70
9. DEN 2.29 2.15
10.LAL 2.09 1.97
11.CHI 1.79 1.61
12.BOS 1.70 1.56
13.IND 1.59 1.43
14.GSW 1.34 1.26
15.MIN 1.28 1.18
16.BRK 0.97 0.86
17.MIL -1.62 -1.51
18.DET -1.62 -1.46
19.UTA -1.87 -1.70
20.ORL -2.58 -2.37
21.POR -2.72 -2.48
22.PHI -2.99 -2.68
23.DAL -3.58 -3.38
24.PHO -4.61 -4.26
25.CLE -5.11 -4.68
26.NOH -5.91 -5.19
27.TOR -5.91 -5.37
28.SAC -7.62 -7.03
29.CHA -9.17 -8.49
30.WAS -10.76 -9.81
Re: Permaximum Ratings and Rankings (Updated Often)
Posted: Fri Dec 28, 2012 2:36 am
by EvanZ
Can you explain how the ratings are "empirically determined"? Also, what does a rating of 30 mean in terms of wins/losses?