APBRmetrics

permaximum wrote:Great. Just when I found out how to do it properly

Could you share your decision on minutes cutoff and cross validation method?

J.E. wrote:

permaximum wrote:Great. Just when I found out how to do it properly

Could you share your decision on minutes cutoff and cross validation method?

You don't need any minute cutoff with ridge (provided you have a reasonable lambda).
As for crossvalidation, I'd randomly remove observations from the data and then "forecast" the left out observations later with the computed ß's. You could do 10-fold CV, but 4-fold already seems enough if you want to save a bit of computing time

RAPM is about twice as accurate as an APM using standard regression and using 3 years of data, where the weighting of past years of data and the reference player minutes cutoff has also been carefully optimized.

permaximum wrote:do you think for 1-year RAPM cutoff isn't needed too?

permaximum wrote:Thanks for all the info again. Then, I won't use minute cutoff for the ridge but qualify players results by minutes for the final results in the end.

I have a simple player box-score metric which I believe near best you can get with raw box scores. I want to check which players' negative or positive effect won't translate to box scores to get a rough idea of those things that box score misses. It will also be useful to see what type of players they are. Then I will decide what to do with the results.

DSMok1 wrote:You might be interested in a couple of articles I wrote: http://godismyjudgeok.com/DStats/2012/n ... -via-rapm/ and http://godismyjudgeok.com/DStats/2012/n ... pm-part-2/

Player                RAPM
1. Nowitzki, Dirk     4.06 (Finals MVP)
2. Garnett, Kevin     3.87
3. Ginobili, Manu     3.73
4. Collison, Nick     3.33
5. Pierce, Paul       3.04
6. Bosh, Chris        2.93
7. Duncan, Tim        2.69
8. James, LeBron      2.63
9. Howard, Dwight     2.57
10.Chandler, Tyson    2.55
...
21.Rose, Derrick      1.91 (MVP)

Player               DRAPM
1. Garnett, Kevin     2.78
2. Brewer, Ronnie     2.52
3. Arthur, Darrell    2.16
4. Duncan, Tim        1.98
5. Allen, Tony        1.96
6. Howard, Dwight     1.77 (DPOY)
7. Bass, Brandon      1.75
8. Pierce, Paul       1.75
9. Livingston, Shaun  1.73
10.Dooling, Keyon     1.69

Player               ORAPM
1. Nowitzki, Dirk     2.49
2. Nash, Steve        2.40
3. Ginobili, Manu     2.40
4. Wade, Dwyane       2.33
5. Smith, J.R.        1.95
6. Collison, Nick     1.90
7. Bonner, Matt       1.85
8. Lawson, Ty         1.77
9. Davis, Baron       1.68
10.Bosh, Chris        1.65

permaximum wrote: Also, I have come to a conclusion that no box score or PBP model (advanced or not) can give better results than RAPM.

permaximum wrote:Box-score metrics should only be used in the beginning of season imo.

permaximum wrote: I can say I'm very satisfied with these results. In the end of each season, I will never look at any player metric but RAPM to evaluate player performance, vote for MVP, DPOY etc. However, in midseason, RAPM won't give very accurate results because of less data. There comes prediction, prior-informed RAPM thus J.E.'s multiyear-RAPM informed RAPM. He says xRAPM is better at that, so I take his word. Towards the end of season, one year uninformed RAPM should give more accurate results than xRAPM as far as seasonal player evaluation goes.

Also, I have come to a conclusion that no box score or PBP model (advanced or not) can give better results than RAPM. In the midseason I would use prior-informed RAPM(xRAPM). For multiple years, RAPM and age-weighted RAPM(are there any?). Towards the end of season and after that, normal RAPM. Box-score metrics should only be used in the beginning of season imo.

mystic wrote:First question: What dataset did you use?

permaximum wrote: Also, I have come to a conclusion that no box score or PBP model (advanced or not) can give better results than RAPM.

How did you come to that conclusion?

In my experience, a blended version of a boxscore-based model and a RAPM model gives the best results in terms of prediction and explanation. When I blended my SPM with Jerry's previously published prior informed RAPM results, I got the best result.

The blend of xRAPM and my SPM showed to be a worse and ended up with a clear bias towards bigger players, which was not seen in such a fashion for a 10 yr dataset of blended SPM+prior informed RAPM.

permaximum wrote:Box-score metrics should only be used in the beginning of season imo.

Boxscore-based models can be as good as a predictor as prior informed RAPM in terms of point differential. The bias towards offense of the boxscore and the lack of informations about team, help and weakside defense makes the differentation between offense and defense rather difficult for individual players. There isn't much of a chance of crediting the correct player with the defensive impact via boxscore in more than 50% of the cases and mostly the rebounder, blocker and the stealer will get the defensive credit. That makes a prediction of the team defensive strength based on individual boxscore-based models basically a coin flip.

But overall you need the boxscore informations, because that is the only way to determine production and efficiency for individual players. And you really need that information, because otherwise you will have a systematic error in the model regarding low and high usage players. You can't just increase the offensive load for an individual player and expect that the efficiency stays the same. But without the boxscore information you don't know who is doing what on the court. RAPM gives you an impact number for players in the used situation. Even if you have some sort of position information added, picking the highest RAPM players for each position may very well not result into the best overall team performance, because you don't know anything about balance and fit; RAPM is not helping you to differentiate players here and does not enable you to find a balanced and fitting lineup.