Yes, I would think that would level the playing field. I haven't seen your ESPN analysis/es. Did all of the tested metrics include a team defensive adjustment? And was that done in a way that ensures the sum of individual players = team point differential (as WP does)?Neil Paine wrote:Among stats with a team adjustment such that the sum of player ratings = team quality, I'm not sure if the heavier DReb weight will lead to more predictive accuracy in the short term. I suppose it's possible, for high-continuity teams in particular, but my assumption was that the defensive team adjustment levels the playing field between metrics in that regard.
Predictive Stats
Re: Translating WS/48 to PER
Re: Translating WS/48 to PER
talkingpractice wrote:In our testing of all of the open source metrics before building our own IPV, prior informed RAPM (using last year RAPM as prior) > ASPM >>>>> everything else (by a mile). Nothing else was really close to those two. And ASPM is as good as RAPM if you're only concerned with the offensive side of the equation (and maybe even better).
IPV is so predictive because it has the benefits of both (prior informed RAPM for defense, and a RAPM model with spm prior for offense).
If considering only open source metrics, then something like RAPMd + 0.75*ASPMo + 0.25*RAPMo would be the nuts.
I don't know why anyone would use offensive WS as opposed to ASPMo.
I continue to feel that some sort of RAPM and statistical plus-minus blend is a pretty good basic choice for today.
Re: Predictive Stats
I agree that team-adjustments are bad (especially on defense), since it basically has to guess at a very high percentage. Effectively un-regressing your highly-predictive regression.
My other theory on why team-adjustments fail (and why I try to never use them) is that the cross-validation/out-of-sample testing (and common sense) shows us that certain stats are more-predictive/more-likely-to-continue than others. Since RAPM (rest its merry soul) uses cross-validation and a huge sample size, these variations are largely weeded out, and resultant a SPM would follow.
But we know, for example, that large variation in opponent Free Throw percentage are largely nulled out in the future. In retrospect, assigning opponent FT% to a player's defense is *mostly* a mistake (i.e. DRTG, WP).
Here is some good out-of-sample stuff:
Ken Pomeroy:
http://kenpom.com/blog/index.php/weblog ... _a_lottery
My article:
http://www.teamrankings.com/blog/ncaa-b ... -geek-idol
Corroborated in theory by Evan's findings:
http://www.d3coder.com/thecity/2012/02/ ... ctor-a4pm/
Now, if we adjusted a team's efficiency margin to regress D3P%, DFT%, etc, accordingly, then we might have a different story. Ever since DRose won MVP I've always wanted to do Min%*TeamEfficiencyMargin (poor Luol Deng).
/endramble
My other theory on why team-adjustments fail (and why I try to never use them) is that the cross-validation/out-of-sample testing (and common sense) shows us that certain stats are more-predictive/more-likely-to-continue than others. Since RAPM (rest its merry soul) uses cross-validation and a huge sample size, these variations are largely weeded out, and resultant a SPM would follow.
But we know, for example, that large variation in opponent Free Throw percentage are largely nulled out in the future. In retrospect, assigning opponent FT% to a player's defense is *mostly* a mistake (i.e. DRTG, WP).
Here is some good out-of-sample stuff:
Ken Pomeroy:
http://kenpom.com/blog/index.php/weblog ... _a_lottery
My article:
http://www.teamrankings.com/blog/ncaa-b ... -geek-idol
Corroborated in theory by Evan's findings:
http://www.d3coder.com/thecity/2012/02/ ... ctor-a4pm/
Now, if we adjusted a team's efficiency margin to regress D3P%, DFT%, etc, accordingly, then we might have a different story. Ever since DRose won MVP I've always wanted to do Min%*TeamEfficiencyMargin (poor Luol Deng).
/endramble