APBRmetrics

The discussion of the analysis of basketball through objective evidence, especially basketball statistics.
It is currently Mon Nov 24, 2014 9:07 pm

All times are UTC




Post new topic Reply to topic  [ 8 posts ] 
Author Message
PostPosted: Fri Apr 15, 2011 5:43 pm 
Offline

Joined: Thu Apr 14, 2011 11:10 pm
Posts: 2548
Page 1

Author Message Ilardi



Joined: 15 May 2008
Posts: 265
Location: Lawrence, KS
Posted: Wed Feb 11, 2009 11:27 am Post subject: Adjusted Plus-Minus Update

I've just calculated mid-season adjusted plus-minus numbers for 2008-09, using the same multi-season methodology that Aaron B. and I used last year to reduce model noise (http://www.82games.com/ilardi2.htm). Specifically, I used all seasons from 2003-2009 (Through Feb 3, 2009), but weighted prior seasons much less heavily (.125 for 07-08, .078 for 03-07 regular seasons, playoffs double). Here are this year's Top 10 on overall APM: Team Player Minutes Offense Defense Error Total CLE James, LeBron 1807 11.31 5.70 1.20 17.02 NOH Paul, Chris 1571 9.51 6.78 1.57 16.29 BOS Garnett, Kevin 1464 6.49 6.93 1.35 13.42 PHI Iguodala, Andre 1858 4.14 8.99 1.27 13.13 MIA Wade, Dwayne 1826 10.70 1.85 1.32 12.55 HOU Ming, Yao 1408 3.49 8.11 1.21 11.60 POR Aldridge, LaMarcus 1770 4.65 5.56 1.39 10.21 DAL Kidd, Jason 1651 4.84 4.65 1.14 9.49 ORL Lewis, Rashard 1807 5.78 3.58 1.09 9.37 SAS Duncan, Tim 1587 4.78 4.22 1.31 9.00 (Sorry about the lack of formatting; I simply pasted straight from an Excel table - I'll be happy to fix it if someone can step me through the process on this interface.) A quick observation: Chris Paul's defensive rating is dramatically higher than it was last year (improving from -4.54 to +6.78 this season). As a result, he now battles LeBron for the greatest overall APM impact. I'd be curious to hear your thoughts on what (if anything) might account for this marked improvement, especially from those of you who have watched the Hornets extensively for the past couple seasons . . .
Back to top

Ryan J. Parker



Joined: 23 Mar 2007
Posts: 711
Location: Raleigh, NC
Posted: Wed Feb 11, 2009 11:31 am Post subject:

Just use the Code button: Code:
Team Player Minutes Offense Defense Error Total CLE James, LeBron 1807 11.31 5.70 1.20 17.02 NOH Paul, Chris 1571 9.51 6.78 1.57 16.29 BOS Garnett, Kevin 1464 6.49 6.93 1.35 13.42 PHI Iguodala, Andre 1858 4.14 8.99 1.27 13.13 MIA Wade, Dwayne 1826 10.70 1.85 1.32 12.55 HOU Ming, Yao 1408 3.49 8.11 1.21 11.60 POR Aldridge, LaMarcus 1770 4.65 5.56 1.39 10.21 DAL Kidd, Jason 1651 4.84 4.65 1.14 9.49 ORL Lewis, Rashard 1807 5.78 3.58 1.09 9.37 SAS Duncan, Tim 1587 4.78 4.22 1.31 9.00
_________________I am a basketball geek.
Back to top

Mountain



Joined: 13 Mar 2007
Posts: 1527
Posted: Wed Feb 11, 2009 12:34 pm Post subject:

What steps? I tried using on one of my excel excerpts in my most recent post (1.Post section. 2. Highlight and click Code.) and it came out all scraggly and I couldn't fix it in repeated tries to adjust it.
Back to top

Ryan J. Parker



Joined: 23 Mar 2007
Posts: 711
Location: Raleigh, NC
Posted: Wed Feb 11, 2009 12:43 pm Post subject:

Well I edited it with vim first. _________________I am a basketball geek.
Back to top

supersub15



Joined: 21 Sep 2006
Posts: 273
Posted: Wed Feb 11, 2009 1:16 pm Post subject:

Ryan J. Parker wrote:
Well I edited it with vim first.
When I copy-paste from Excel, cell delimiters are replaced with tabs. I usually replace the tabs with spaces, then add the code tag. Is there a way to switch tabs to spaces automaticlly in VIM without having to do the alignment manually afterwards?
Back to top

Ilardi



Joined: 15 May 2008
Posts: 265
Location: Lawrence, KS
Posted: Wed Feb 11, 2009 1:19 pm Post subject:

Back to the question of CP3's defensive improvement . . . Last year, Paul's opposing PGs had an eFG% of 51.8%; this year it's down to 46.7%. Likewise, this year opposing PG assists are down, turnovers up, iFG% down, and PER down. All that with Chandler missing major minutes this year (and playing much less effectively on d when he's been in).
Back to top

Ryan J. Parker



Joined: 23 Mar 2007
Posts: 711
Location: Raleigh, NC
Posted: Wed Feb 11, 2009 1:32 pm Post subject:

supersub15 wrote:
Ryan J. Parker wrote:
Well I edited it with vim first.
When I copy-paste from Excel, cell delimiters are replaced with tabs. I usually replace the tabs with spaces, then add the code tag. Is there a way to switch tabs to spaces automaticlly in VIM without having to do the alignment manually afterwards?
You might try :set ts=10 Or some other large number. So when you copy/paste you get the spaces that you want._________________I am a basketball geek.
Back to top

Mountain



Joined: 13 Mar 2007
Posts: 1527
Posted: Wed Feb 11, 2009 1:55 pm Post subject:

It is a dramatic turnaround. I don't have deep first-hand knowledge on the situation. Wonder how much is related to Posey. What's his defensive adjusted in New Orleans, single year or under the new weights? Or maybe more importantly did the departure of Pargo and Jackson (good on raw team defense on/off) help Paul rate better- without those comparisons? Did the end of the two PG guard sets make Paul perform better?
Back to top

Ilardi



Joined: 15 May 2008
Posts: 265
Location: Lawrence, KS
Posted: Wed Feb 11, 2009 2:40 pm Post subject:

This season: Posey on offense: -0.16 Posey on defense: -1.43 I like the hypothesis about the end of the 2-pg sets this year giving CP a big boost on d, as it means he's no longer trying to guard 2s who are often 6+ inches taller.
Back to top

bastillon



Joined: 04 Nov 2008
Posts: 55
Posted: Wed Feb 11, 2009 5:42 pm Post subject:

Ilardi wrote:
This season: Posey on offense: -0.16 Posey on defense: -1.43 I like the hypothesis about the end of the 2-pg sets this year giving CP a big boost on d, as it means he's no longer trying to guard 2s who are often 6+ inches taller.
I wonder how is that ? Which way can he hurt the team ? Efficient shooter, making place in the middle and great team defender, especially on rotations and mismatches. Given that he's Peja's replacement it's almost impossible he's makin a bad effect on team defense. Can it be caused by the sets of players he's on the floor with or is it already in your adjusted formula ?
Back to top

dudeimstoked



Joined: 15 Oct 2007
Posts: 24
Posted: Wed Feb 11, 2009 11:20 pm Post subject:

bastillon wrote:
I wonder how is that ? Which way can he hurt the team ? Efficient shooter, making place in the middle and great team defender, especially on rotations and mismatches. Given that he's Peja's replacement it's almost impossible he's makin a bad effect on team defense. Can it be caused by the sets of players he's on the floor with or is it already in your adjusted formula ?
maybe he's overrated. adjusted attempts to remove the other players that are on the floor with him._________________http://www.apbrmetrics.com http://www.puresabermetrics.com
Back to top

Mountain



Joined: 13 Mar 2007
Posts: 1527
Posted: Thu Feb 12, 2009 3:13 am Post subject:

The 2 PG thing is probably part of it but Paul probably is playing better defense too. Hungry to advance. Posey is near enough to neutral on both offense and defense which is not bad for a 6th man.
Back to top

Mike G



Joined: 14 Jan 2005
Posts: 3596
Location: Hendersonville, NC
Posted: Thu Feb 12, 2009 10:26 am Post subject:

Sorry if it's been clarified already, but some questions still, about Adj+/- : - Is 'garbage time' discounted at all? I know Dan R adopted a pretty strict policy about it, early on. - It was mentioned that a possession doesn't count if it's less than 5 (?) seconds to play in a period. Otherwise it is considered to be a full possession? Would it be better to call 12 sec .50 of a poss, 3 sec = .125 poss, etc? - What about 'special situation' lineups: At the end of a period, you may put in 5 shooters, or 5 defenders. Especially in combination with the (few seconds) situation above, defenders may look pretty good on the average such possession. And the shooters/ballhandlers may look pretty bad._________________` 36% of all statistics are wrong
Back to top

bchaikin



Joined: 27 Jan 2005
Posts: 688
Location: cleveland, ohio
Posted: Thu Feb 12, 2009 11:57 am Post subject:

Chris Paul's defensive rating is dramatically higher than it was last year (improving from -4.54 to +6.78 this season).... I'd be curious to hear your thoughts on what (if anything) might account for this marked improvement... wow - so he's dramatically better on defense now than he was just a year ago because adjusted +/- says so?... how about looking at some new orleans hornets / chris paul on court/off court numbers for last year and this year (courtesy 82games.com): here are hornets numbers with paul on court: -----year--------------0708-----0809 pts/100poss off----116.0----115.0 pts/100poss def---107.6----106.1 --difference--------- +8.4 ---- +8.9 --min played-------- 3004 ---- 955 i don't see a whole lot of difference here from last season to this season - the offense is a bit worse now than last year, and the defense a bit better. now let's look at the hornets with paul off the court: -----year--------------0708-----0809 pts/100poss off----100.6----- 98.9 pts/100poss def---101.4----113.5 --difference---------- -0.8 ---- -14.6 --min played-------- 1685 ----- 570 whoa - big difference here. without chris paul on the floor the hornets are much worse on defense now than just a year ago when paul was not on the floor. they allowed 101.4 pts/100poss without paul in 07-08, but a much worse 113.5 pts/100poss without paul now... in 07-08 with paul their def pts/100poss allowed was 107.6, but much better without paul at just 101.4 pts/100poss allowed. last year in the chris paul thread you tried to convince us that paul was such a poor defender because of this that his poor defense for all intent and purpose negated his excellent offense... but now a year later in 08-09 they are at 106.1 pts/100poss allowed with paul, better but not by much than in 07-08 at 107.6, yet without paul the team is much worse on defense now than a year ago at 113.5 pts/poss allowed per 100 possesions... so because of what happened when chris paul was not on the floor both seasons you are trying to say that paul's defense is dramatically better now than a year ago, when what has occured when he was actually on the floor is not dramatically different than a year ago?... how does what happened when paul did not in fact play affect his defense dramatically?... the hornets went 56-26 last year in 07-08 and were 11th in the league in defense (lowest pts/poss allowed) and paul led the team in minutes played... so far in 08-09 the hornets are on pace to go about 49-33 to 50-32, and the team is 13th in the league in defense, and paul is again leading the team in minutes played... and the hornet's pts/poss on offense (116.0/115.0) and defense (107.6/106.1) are about the same each season with paul on the floor, and paul's stats across the board this year are quite similar to last year (shooting a bit better, almost 3%, rebounding about 25% better now)... paul was a better than average to good defender in 07-08 (actually got the most votes among PGs for the all-D team, more than either billups or kidd), primarily because of his high steals, and so far in 08-09 is a better than average to good defender, again primarily because of his high steals...
Back to top

Mountain



Joined: 13 Mar 2007
Posts: 1527
Posted: Thu Feb 12, 2009 12:45 pm Post subject:

Good add to the analysis Bob. I looked at Paul at counterpart level and in player pairs for current vs. last season for differences. I also looked at the on/off but I guess I failed to compare equivalent parts season to season. It does look to me that the off data is having a significant impact on the adjusted estimate, maybe too much. Here the very strong weight for the current season may perhaps be amplifying Paul's defensive number. This is a challenging task.

Page 2

Author Message Mountain



Joined: 13 Mar 2007
Posts: 1527
Posted: Thu Feb 12, 2009 2:22 pm Post subject:

I came back across Ryan's Points Added again. http://tinyurl.com/cfclf9 I found myself thinking there are some advantages of a positive 0-1 scale. Focus on the scoreboard producing positive rather than trying to pin down blame at the same time or really wrestle with the causation/correlation question about that "blame". Of course data is lost too. This cut is probably more similar to statistical +/- than pure adjusted. I'd be hearing more from Ryan about his model and comparison of the results to adjusted and comment from the adjusted modelers on it. And from anyone about the possible value of running a negative scale -1 to 0. Doing so would give another view different from what the positive scale does (does this essentially focus on the non-boxscore?) and similar to what adjusted does to get offense/defense splits. And if you still split out offense and defense you could see how many points added and subtracted a player causes on offense and defense instead of just the net so you could see who still has work to do minimizing negatives even with their positives on a side of play or recognize the low mistake guys even if they aren't doing the positive work (sometimes they don't have to with others in roles and with high abilities for that.) Having pure adjusted, statistical and points added positive and negative would give at least 4 perspectives. In the end it is about gathering information for consideration, multi-dimensional (matter/energy) understanding building and not one number settles everything.
Back to top

Ryan J. Parker



Joined: 23 Mar 2007
Posts: 707
Location: Raleigh, NC
Posted: Thu Feb 12, 2009 2:37 pm Post subject:

As DLew pointed out, my formulation is really just a translation of +/-. The estimates are clearly going to be different, but for the most part it's the same thing. The published work on adjusted +/- doesn't spell out the weighting very clearly (consider this my naivety with fitting the data with a classical linear regression model), so I went this route to try and grasp the concept. Also, See the latest discussion on Chris Paul to see the difficulties underlying the affects of a player's coefficients. Does the methodology really do a good job of rating players versus the rest of the league? Lets suppose we rate a player's rebounding %. Is it safe to assume that players playing on teams with non-normal rebounding strategies are worse than the other players in the league? How do we measure this affect? I mean clearly they go smaller which has an affect, but I'd think all that leaking out has an important role as well. There are some important things we don't have a good grasp on that affect these ratings. That is what I'm studying. _________________I am a basketball geek.
Back to top

Mountain



Joined: 13 Mar 2007
Posts: 1527
Posted: Thu Feb 12, 2009 3:32 pm Post subject:

Ok "my formulation is really just a translation of +/-". A statistical player translation of raw team +/-? I think that is what you mean. right? If so then a -1 to 0 scale isn't going to work here without a shot defense blame assignment. And total blame by matchup isn't going to justify and other split rules aren't going to justify many. 0 to 1 and -1 or 0 scale runs with the adjusted method could be done though, I think and might be worthing do, I think.
Back to top

Ilardi



Joined: 15 May 2008
Posts: 262
Location: Lawrence, KS
Posted: Thu Feb 12, 2009 10:43 pm Post subject:

bchaikin wrote:
Chris Paul's defensive rating is dramatically higher than it was last year (improving from -4.54 to +6.78 this season).... I'd be curious to hear your thoughts on what (if anything) might account for this marked improvement... wow - so he's dramatically better on defense now than he was just a year ago because adjusted +/- says so?... how about looking at some new orleans hornets / chris paul on court/off court numbers for last year and this year (courtesy 82games.com): here are hornets numbers with paul on court: -----year--------------0708-----0809 pts/100poss off----116.0----115.0 pts/100poss def---107.6----106.1 --difference--------- +8.4 ---- +8.9 --min played-------- 3004 ---- 955 i don't see a whole lot of difference here from last season to this season - the offense is a bit worse now than last year, and the defense a bit better. now let's look at the hornets with paul off the court: -----year--------------0708-----0809 pts/100poss off----100.6----- 98.9 pts/100poss def---101.4----113.5 --difference---------- -0.8 ---- -14.6 --min played-------- 1685 ----- 570 whoa - big difference here. without chris paul on the floor the hornets are much worse on defense now than just a year ago when paul was not on the floor. they allowed 101.4 pts/100poss without paul in 07-08, but a much worse 113.5 pts/100poss without paul now... in 07-08 with paul their def pts/100poss allowed was 107.6, but much better without paul at just 101.4 pts/100poss allowed. last year in the chris paul thread you tried to convince us that paul was such a poor defender because of this that his poor defense for all intent and purpose negated his excellent offense... but now a year later in 08-09 they are at 106.1 pts/100poss allowed with paul, better but not by much than in 07-08 at 107.6, yet without paul the team is much worse on defense now than a year ago at 113.5 pts/poss allowed per 100 possesions... so because of what happened when chris paul was not on the floor both seasons you are trying to say that paul's defense is dramatically better now than a year ago, when what has occured when he was actually on the floor is not dramatically different than a year ago?... how does what happened when paul did not in fact play affect his defense dramatically?... the hornets went 56-26 last year in 07-08 and were 11th in the league in defense (lowest pts/poss allowed) and paul led the team in minutes played... so far in 08-09 the hornets are on pace to go about 49-33 to 50-32, and the team is 13th in the league in defense, and paul is again leading the team in minutes played... and the hornet's pts/poss on offense (116.0/115.0) and defense (107.6/106.1) are about the same each season with paul on the floor, and paul's stats across the board this year are quite similar to last year (shooting a bit better, almost 3%, rebounding about 25% better now)... paul was a better than average to good defender in 07-08 (actually got the most votes among PGs for the all-D team, more than either billups or kidd), primarily because of his high steals, and so far in 08-09 is a better than average to good defender, again primarily because of his high steals...
Bob, In response: 1) With all due respect, it appears you've mischaracterized my position. In the thread on CP3's defense last year, I noted that Paul rated quite highly (25th best overall) in my adjusted plus-minus analysis of 06-07 . . . and in light of this fact, I speculated on the likelihood that his abysmal defensive APM rating in Eli's 07-08 model was perhaps due in part to the rather noisy estimate (i.e., high error on CP3's defensive rating). Yes, I still concluded that he was a liability on defense in 07-08 (a position further corroborated by the much less noisy 5-year APM model I published with Aaron B. on 82games this year), but not to the extent that this completely "negated" his off-the-charts offensive contributions. 2) Further, I didn't say Chris Paul is "dramatically better on defense now"; what I said was "his defensive rating is dramatically higher". It's perhaps a subtle point, but an important one. One might infer the former from the latter, but a better interpretation would be "his overall impact on the Hornets' defensive efficiency is markedly more positive this year." 3) Your analysis regarding team offensive/defensive efficiency with Paul on-vs-off court does not strike me as persuasive. Simply put: since the Hornets are considerably worse on defense with Paul off-court this year in comparison with last year, it is parsimonious to assume (in the absence of evidence to the contrary) that Paul's teammates this season are playing poorer aggregate defense all the time, including all those minutes when Paul is on the court. And if this is indeed the case, then there is only one way to account for the fact that the Hornets have improved in defensive efficiency with Paul on-court this year in comparison with last: Paul is having a more positive impact. 4) Evidence consistent with this interpretation is found in the fact (noted earlier) that Paul's opposing PGs have an eFG% that's dropped from 51.8% last year to 46.7% this season. Likewise, this year opposing PG assists are down, turnovers up, iFG% down, and PER down. All that with Chandler missing major minutes this year (and playing much less effectively on d when he's been in). Thus, it does not strike me as implausible to assert that Paul is having a considerably more positive impact on the Hornets' defensive efficiency this year. Defense is largely about effort, and it's not unusual for high-usage offensive stars to "pick their moments" on d, i.e., to conserve energy on the defensive end (Brandon Roy strikes me as a good example, as did CP3 when I watched him last season). But some such players ultimately decide to take it to the next level on d: LeBron has been highly vocal this season on that very point, and his markedly improved defensive APM backs him up. My sense is that Paul has done likewise this season, but having watched the team only a couple times this year, I cannot verify it. Hence, my earlier query . . .
Back to top

bchaikin



Joined: 27 Jan 2005
Posts: 681
Location: cleveland, ohio
Posted: Fri Feb 13, 2009 2:39 am Post subject:

With all due respect, it appears you've mischaracterized my position. did you or did you not start this thread with this statement: Chris Paul's defensive rating is dramatically higher than it was last year (improving from -4.54 to +6.78 this season). and did you or did you not reference your article on 82games.com for 07-08, in which chris paul and his defensive adjusted +/- -4.54 rating is listed at the bottom of the defensive adjusted +/- list among other supposed poor defenders like michael redd and al jefferson? i'm assuming a high negative number there represents a poor defender and a high positive number represents a very good defender - or is this a bad assumption?... in the "chris paul +/-" thread you also made this statement: Bottom line: the Hornets give up more points with Paul on the court than they do without him (over 6 more points per 100 possessions). Because this on-off value is the worst on the team, it does not appear to be a mere artifact of his playing with other weak defenders, nor does it appear to be a fluke of sampling (since his on-off defensive rating was virtually identical during the 06-07 season: -6.8 pts/100 possessions). despite the fact that chris paul played 3 times as many minutes compared to what the hornets played without him, you insisted his defense was poor based on what the team did without him... and did you or did you not in the "new adjusted plus minus ratings for 07-08" thread make this statement: Although Chris Paul was one of the Top 3 offensive players in the league (+9.24), he was actually a liability on the defensive side of the ball (-4.54). As a result, his overall contribution (+4.69), while impressive, was not commensurate with that of a top MVP candidate. saying in no uncertain terms that the PG with the most votes for the nba all-D team was a poor defender (or does "liability on the defensive end" mean something other that what it sounds like - i'd hate to mischaracterize you here), and that a player that was voted to the all-D 2nd team and the all-nba 1st team was not a viable mvp candidate?... and now you make this statement: Further, I didn't say Chris Paul is "dramatically better on defense now"; what I said was "his defensive rating is dramatically higher". It's perhaps a subtle point, but an important one. One might infer the former from the latter, but a better interpretation would be "his overall impact on the Hornets' defensive efficiency is markedly more positive this year.... there is only one way to account for the fact that the Hornets have improved in defensive efficiency with Paul on-court this year in comparison with last: Paul is having a more positive impact..." in the span of a few paragraphs you have gone from "dramatically" to "markedly more positive" to just "more positive". so just how big is a jump from -4.54 to +6.78 in defensive adjusted +/-? is that an awful to an excellent defender, a poor one to a very good one, or is it just a below average defender to a just above average defender? your 07-08 listing shows only one player better than +6.78 (kevin garnett at +7.59) in adjusted defensive +/- and only four players with a rating less than -4.54 (al jefferson, al thornton, leandro barbosa, and kevin martin) out of a list of some 120+ players who played over 2000 minutes. so is this range of paul's from 07-08 to 08-09 very large or not?... Your analysis regarding team offensive/defensive efficiency with Paul on-vs-off court does not strike me as persuasive. it wasn't meant to be persuasive, just factual. those numbers came straight off the 82games.com website... Evidence consistent with this interpretation is found in the fact (noted earlier) that Paul's opposing PGs have an eFG% that's dropped from 51.8% last year to 46.7% this season. perhaps you should re-read the "chris paul +/-" thread... let's do the math again. the league average PG in 07-08 playing about 3000 minutes took about 1000 shots, so the average PG playing 3000 minutes defended about 1000 shots. the difference between defending at 46.7% vs 51.8% would be 467 vs 518 FGM or 51 FGM over a season. if his improvement this year is in forcing another 51 missed FGAs over the span of a season, then those missed FGAs have to be rebounded by the defense to be a defensive stop. assuming 70% are rebounded by the defense that's about 51 x .70 = 36 more defensive stops over the span of a season due to better defense on FGAs... but we are talking about a player who got 217 steals in 07-08 (each of which is a defensive stop) and is on pace to get another 215-220 steals this season. that's 215/36 or about 6 times as many defensive stops due to steals than the additional stops he forces due to better defense on shot attempts... so you can quote those opposing eFG%s all you want, but we are talking about a player here who gets a ton of defensive stops from steals that dwarfs any difference in his eFG% allowed from last season to this season - in other words a player who was a good defender both last year and this year... Likewise, this year opposing PG assists are down, turnovers up, iFG% down, and PER down. again last season the PER for hornets opposing PGs when paul was on the floor was 18.5, but in 08-09 its 16.9. i don't see alot of difference here. better defensively? yes, but not by much, nothing that is "dramatic" or "markedly more positive"... it does not strike me as implausible to assert that Paul is having a considerably more positive impact on the Hornets' defensive efficiency this year. where's the evidence? adjusting a player's "defense" dramatically (saying he's a liability one season but then markedly more positive the next) based on what happened when he did not play, even though the team defense when he did in fact play changed little, does not seem like plausible evidence at all... "...some such players ultimately decide to take it to the next level on d: LeBron has been highly vocal this season on that very point, and his markedly improved defensive APM backs him up. My sense is that Paul has done likewise this season, but having watched the team only a couple times this year, I cannot verify it. Hence, my earlier query..." are you seriously trying to equate paul's supposed better defense this season compared to last with the better defense of lebron james from last year to this year? the hornets allowed 107.6 pts/100poss with paul in 07-08, 106.1 pts/100 poss with paul so far in 08-09. that's a difference of just 1.5 pts/100poss, enough to move a team up the defensive rankings about 1-5 notches in rank... however in 07-08 with james cleveland allowed 107.7 pts/100poss, yet so far in 08-09 just 100.8 pts/100poss. that's a difference of almost 7 pts/100poss, which right now is the difference in defensive ranking between the best defensive team in the league and about the 18th best (13th worst) team in the league defensively in terms of pts/100poss allowed... you might want to try another more analogous comparison for paul other than james...
Back to top

DLew



Joined: 13 Nov 2006
Posts: 222
Posted: Fri Feb 13, 2009 3:05 am Post subject:

Bob, Let's discuss the 82games information which you provided, as I feel that you did not make the appropriate inference. (1) The Hornets are much worse on defense this year with Chris Paul off the court, as compared to last year with Chris Paul off the court. From this we learn that players on the Hornets not named Chris Paul are collectively playing worse defensively this year than last year. (2) The Hornets are better on defense this year with Chris Paul on the court as compared to last year with Chris Paul on the court. Given that we know that his teammates are playing worse defensively than last year, this piece of information strongly suggests that Chris Paul is playing far better defensively. Please let me know if you feel this line of reasoning is valid. It is the one that the 82games numbers suggest to me, but obviously leads to a different conclusion than the one you came to.Last edited by DLew on Fri Feb 13, 2009 11:25 am; edited 1 time in total
Back to top

cherokee_ACB



Joined: 22 Mar 2006
Posts: 157
Posted: Fri Feb 13, 2009 6:57 am Post subject: Re: Adjusted Plus-Minus Update

Ilardi wrote:
I'd be curious to hear your thoughts on what (if anything) might account for this marked improvement, especially from those of you who have watched the Hornets extensively for the past couple seasons . . .
It's probably because the Hornets have been successful so far this year by going small, at least on defense. Which is Devin Brown's defensive rating? Must be sky high.
Back to top

deepak



Joined: 26 Apr 2006
Posts: 664
Posted: Fri Feb 13, 2009 10:42 am Post subject:

DLew wrote:
Please let me know if you feel this line of reasoning is valid. It is the one that the 82games numbers suggest to me, but obviously leads to a different conclusion that which you came to.
I think Bob's inference assumed that the players on the Hornets roster can be divided into two sets -- one that plays almost exclusively with Paul, and another that plays almost exclusively without him. If that's how we look at it, then Paul's raw Off-Court +/- is essentially independent of his On-Court +/- (since it involves more or less different players). So to judge a player's defensive prowess based on +/-, you would just need to look at the On-court numbers relative to league average. Your inference assumes that the mixture of players Paul plays with is very similar to the mixture of players that are on the floor when he's sitting. So, his raw Off-Court becomes a reflection of the team's defense, minus Chris Paul. Looking at it that way, taking the difference between On-court and Off-Court ("Roland Rating") would be a better way to judge a player's defensive impact. I think the reality would lie somewhere in between. Maybe one could come up with a quick and dirty Defensive APM, discarding opponents faced, by using: est_DAPM = x * (On-Court_Def - LgAvg_Def) + (1-x)*(On-Court_DF - Off-Court_Def), with x being between 0 and 1 If we favor Bob's interpretation more, then we put x closer to 1. If we favor your interpretation more, we put it closer to 0.
Back to top

Mountain



Joined: 13 Mar 2007
Posts: 1527
Posted: Fri Feb 13, 2009 1:50 pm Post subject:

In rough terms Paul on the court is with West 80% of the time, Peja 70%, Butler 60%, Posey 50% and Chandler 50%. The rest of team fills 90% of one slot. Paul off the court has 85% Posey, 45% West, 60% Butler, 60% Daniels, 50% Brown, 50% Armstrong, 30% Marks, 30% Wright, just a bit over 10% for Chandler, 20% Peterson and 40% other. So Paul has 6 pretty solid pieces on team defense for about 78% of the available total time. The team with Paul off plays those same guys just 50% of total time. That edge in having familiar decent pieces around him on defense probably helps Paul. Maybe B Scott needs to sort out a non-Paul roster that does better on defense. Maybe that group needs more continuity than 10+ guys checking in and out. Maybe they need more Chandler or a better Chandler substitute or a better non-Chandler defensive scheme. Or better backcourt protection / court leadership without Paul. Needs something(s).
Back to top

bastillon



Joined: 04 Nov 2008
Posts: 55
Posted: Fri Feb 13, 2009 3:18 pm Post subject:

Quote:
where's the evidence? adjusting a player's "defense" dramatically (saying he's a liability one season but then markedly more positive the next) based on what happened when he did not play, even though the team defense when he did in fact play changed little, does not seem like plausible evidence at all...
Look at it this way: Chandler as a centerpiece of their defense last year playing much less and being even less productive and they still manage to be better by 1.5 pts/100poss. That, as told already, suggests that Paul is really playing better defense. Hadn't he played better D, Hornets would make a dramatic fall in all the ratings. That also shows how hard it is to make these ratings based on few seasons data, because you have to analyze every single player with his defensive improvement. Defensive improvement screws APM totally. Of course you can change values of weighs and all you want, but actually a player can have a defensive turnaround(lets say someone's got potential, but he doesn't give enough effort, while next season his intensity makes him one of league-elite like LeBron) in one season. Now to discover what's going on we've gotta make few-page discussion. And now even better example: what would your ratings show if the whole team turned around from one of the league's laziest defense to TOP10 like Nuggets this year ? One could have wrong impressions about Billups factor(or one of you really believe this crap ?). What's Nuggets 2-year adj. +/- ? Especially defensively. I'm still wondering about that Posey guy and what I'm learning from all the facts we know is that there's certainly a bad weight in your APM. If guy cannot hurt the team in any way(better than the rest of Hornets SFs on D, high percentage shooter on the other end) and still one could make an arguement he's 'overrated' something's wrong. Not enough weight of teammates impact.
Back to top

Ilardi



Joined: 15 May 2008
Posts: 262
Location: Lawrence, KS
Posted: Sun Feb 15, 2009 12:42 pm Post subject:

There's much to reply to in recent posts, so I'll do my best to proffer a response on the most salient points: 1) Chris Paul's defensive APM (adjusted plus-minus) has improved markedly over last season, and there are converging lines of evidence regarding the validity of these observed APM changes, including: (a) the improved on-court defensive efficiency of the 08-09 Hornets (versus 07-08 Hornets) with Paul on-court, despite marked deterioration in the team's defensive efficiency with Paul off-court (and, as Mountain noted, the 6 players most commonly on-court with Paul in 08-09 are still on-court 50% of the time when Paul is out, so there is substantial personnel overlap across the team's Paul-on and Paul-off minutes); (b) an improved 08-09 Hornets defensive efficiency with Paul on-court despite the stark drop in effectiveness (and minutes) of the team's 07-08 defensive centerpiece, Tyson Chandler; and (c) a substantial drop in the effectiveness of Paul's opposing PGs in 08-09. (Bchaikin attempts to dismiss this latter point, waving away, for example, the impact of the drop in eFG% - down to 46.7% from last year's 51.8% - by calculating a net of roughly 36 more defensive stops per season, but he ignores the fact that much of a player's defensive efficiency lies in his contribution to team defense - rotations, double-teams, transition coverage, traps, etc.; if a player's on-ball defense is improved, presumably it's not a stretch to infer that his off-ball defense has experienced commensurate improvements; bchaikin might also ponder what Paul's opposing PGs would have yielded in collective FG% this season had the Hornets benefited from the 07-08 version of Chandler.) 2) Bastillon raises interesting questions about the validity of defensive APM ratings when they're derived from a multi-season database, given the apparently high potential for variability in individual performances from one season to the next. My own take is that there's one sense in which the concern is warranted, and one in which it's not. Let me explain. My primary goal in APM analyses is to provide a metric of a player's actual contribution to the team's bottom line in a given season, not to provide an indicator of a player's potential contribution (e.g., if he were to start playing harder, adopt a different role in a different system, etc.). What I'm discovering is that offensive APM (which correlates quite heavily with traditional boxscore stats, and metrics like PER that are derived from them) is more stable from year to year than is defensive APM. This isn't terribly surprising, but I think it's an important discovery with myriad implications. It suggests, for example, that many players have the potential to greatly increase their on-court impact by giving greater effort (and playing smarter) on the defensive side of the ball. Now, is it a problem for the APM analysis that some players (like Paul and LeBron) experience major changes in defensive APM from one season to the next? Not really. The model uses 6 seasons' worth of data on the premise that cross-season regularities in performance (which decidedly exist) help reduce estimation error for any given season, and this is simply an empirical question that has been answered in the affirmative. For example, the standard error in Paul's 08-09 defensive APM estimate (through ~50 games) is only 1.57 (using 03-08 seasons, with very low weightings, to reduce model error), compared with an implied standard error for Paul's single-season 07-08 estimate (in Eli W's single-season analysis) of over 7.0. In other words, using multiple seasons' worth of data, we can still get low-noise estimates of each player's performance in any given season, despite the fact that some players will vary quite a bit on the defensive end from year to year. Nevertheless, as Bastillon implies, it would be a mistake to look at a player's defensive APM number from a given season and conclude: this is a valid rating of Player X's defensive ability. Rather, it's merely an indicator of his defensive contribution over the period in question. 3) Bastillon also wonders about the apparent threat posed by a whole-team turnaround in defensive effort, and he proffers the 08-09 Nuggets as Exhibit A. Well, first, the Nuggets weren't that bad defensively in 07-08 (in fact, they were above average). They've improved from #12 in the league in 07-08 (106.2 ppg) to #6 this season (102.4) - a nice jump, but hardly a revolutionary metamorphosis. Regardless, the APM model can handle such cross-season fluctuations just fine: it merely reflects the fact that some players (perhaps most) on a given team experience improvement in defensive APM from one season to the next. While such team-wide improvements might add a bit more noise to the overall model, the effect is not terribly large - and besides, it's an effect that's readily observable in the standard error terms associated with each player's estimates, so the model will actually tell us if there's a problem worth worrying about. Ok, there's lots more to say, and more questions I haven't yet addressed, but I need to go grade some papers, so hopefully this is at least a decent start . . . more anon. P.S. Bastillon, I'm having trouble interpreting your question about Posey: perhaps you could re-state it? For what it's worth, I have his defensive APM dropping from 0.57 last year in Boston (slightly above-average defensive contribution) to -1.43 this season. This drop is about what we might expect of a 32-year-old player (i.e., past his prime) adjusting to a new system.
Back to top

bchaikin



Joined: 27 Jan 2005
Posts: 681
Location: cleveland, ohio
Posted: Sun Feb 15, 2009 1:23 pm Post subject:

...this piece of information strongly suggests that Chris Paul is playing far better defensively. to whom? to you? the hornets team defense gets a bit better (not insignificant, but not significantly better) when paul was on the floor from one season to the next (107.6 to 106.1 pts/poss allowed), but because of something that has absolutely nothing to do with chris paul himself, and based on a sample size of data that is just 1/3 the size of the data that when paul played and is thus much more apt to be skewed, adjusted +/- comes to the conclusion that paul "...was actually a liability on defense..." one season (07-08 ) but is "...far better defensively..." the next (08-09)?... this despite the fact that the defensive pts/poss allowed when he actually did play changed little? and despite the fact that the 07-08 season that adjusted +/- finds paul to be a liability on defense there are many others who have found his defense to be quite good? after all he received not only the most votes among all PGs for the all-D team, but also the 7th most votes among all players in the entire league for DPOY? and this from the people that have actually seen him play alot - how much did you actually see chris paul play last year?... tell us then - knowing that the number of steals paul got both seasons is about the same (and each is a defensive stop), and the data for eFG% allowed seems to show at most 1 additional defensive stop through missed FGAs forced every 2 games, just what exactly has chris paul - himself - done this year that he did not do last year such that he is "...playing far better defensively..."?... do you have any evidence whatsoever that paul has forced significantly more non-steal turnovers, or that he was personally responsible for forcing more missed FGAs by the players guarded by his teammates (that might not be reflected in data such as that found on 82games.com) this year than last year?... "...a substantial drop in the effectiveness of Paul's opposing PGs in 08-09. Bchaikin attempts to dismiss this latter point, waving away, for example, the impact of the drop in eFG% - down to 46.7% from last year's 51.8% - by calculating a net of roughly 36 more defensive stops per season, but he ignores the fact that the bulk of a player's defensive efficiency lies in his contribution to team defense - rotations, double-teams, transition coverage, traps, etc.; if a player's on-ball defense is improved, presumably it's not a stretch to infer that his off-ball defense has experienced commensurate improvements, as well;..." oh, i get it. let's just completely ignore the actual data that does in fact exist, and then infer what we will to validate our conclusions. i'll ask again, how much have you actually seen chris pual play? because in the opinions of those who have seen him play the most (the coaches and media) he was a very good defender in 07-08... "...bchaikin might also ponder what Paul's opposing PGs would have yielded in collective FG% this season had the Hornets benefited from the 07-08 version of Chandler...." oh, so now not only something that happened when chris paul was not on the floor, but something that has not happened period, is futher validation of adjusted +/- conclusions?...
Back to top

Neil Paine



Joined: 13 Oct 2005
Posts: 774
Location: Atlanta, GA
Posted: Sun Feb 15, 2009 1:28 pm Post subject:

Ilardi wrote:
My primary goal in APM analyses is to provide a metric of a player's actual contribution to the team's bottom line in a given season, not to provide an indicator of a player's potential contribution (e.g., if he were to start playing harder, adopt a different role in a different system, etc.) ... as Bastillon implies, it would be a mistake to look at a player's defensive APM number from a given season and conclude: this is a valid rating of Player X's defensive ability. Rather, it's merely an indicator of his defensive contribution over the period in question.
Personally, I like APM, and I believe there's a lot of potential there as APBRmetrics moves into the future. But I guess the question a lot of people would have, given what I quoted, is "what's the point of APM?" I mean, if I'm a GM, I'm probably not asking how good a player was in the past, I'm asking how good he is now -- and more importantly, what can I expect out of him in the future? This is one of those "value vs. ability" questions we always run into, and I've found that while "ability" is the real practical concern for front offices, "value" is one of those almost purely academic exercises regarding past players that's fun for discussion, but of little use to decision-makers in the here and now (besides, in APM's case, it can't even describe the past value of players before PBP was widely available). So I suppose the question is, if APM doesn't really try to predict future performance, and it can't tell us about past performance except over the past handful of seasons, doesn't that kind of limit its utility? (Just playing devil's advocate here, btw)
Back to top

Neil Paine



Joined: 13 Oct 2005
Posts: 774
Location: Atlanta, GA
Posted: Sun Feb 15, 2009 1:44 pm Post subject:

bchaikin wrote:
the hornets team defense gets a bit better (not insignificant, but not significantly better) when paul was on the floor from one season to the next (107.6 to 106.1 pts/poss allowed), but because of something that has absolutely nothing to do with chris paul himself, and based on a sample size of data that is just 1/3 the size of the data that when paul played and is thus much more apt to be skewed, adjusted +/- comes to the conclusion that paul "...was actually a liability on defense..." one season (07-08 ) but is "...far better defensively..." the next (08-09)?...
At the same time, I don't understand why you're having so much trouble understanding the concept behind on/off +/- metrics, Bob... When a player is on the court, the team's DRtg is not only a function of his ability, but that of 4 other guys. So how do you isolate his specific contribution? You find instances in which some combination of the other 4 guys are on the court, and the player we're studying isn't. Is the team's defense better? That suggests the other 4 guys were contributing more to the team's DRtg when our guy was on the court (which is essentially what happened in '08: NOH was 6.2 efficiency points worse on D when Paul was in the game vs. when he wasn't). Or is the defense a lot worse? That suggests our guy was actually making a far bigger contribution than the other 4 guys when he was on the floor (like Paul in '09: NOH is 5.7 DRtg points better when he's on vs. off). It's really not a difficult concept to grasp on paper -- the hard part is getting around the fact that players often play with generally the same group of players, because it's tough to isolate one specific player vs. the other 4 guys on the court with him when he's always playing with the same 4 guys, and they're always sitting when he's sitting.
Back to top

Kevin Pelton
Site Admin


Joined: 30 Dec 2004
Posts: 976
Location: Seattle
Posted: Sun Feb 15, 2009 1:53 pm Post subject:

Ilardi wrote:
What I'm discovering is that offensive APM (which correlates quite heavily with traditional boxscore stats, and metrics like PER that are derived from them) is more stable from year to year than is defensive APM. This isn't terribly surprising, but I think it's an important discovery with myriad implications. It suggests, for example, that many players have the potential to greatly increase their on-court impact by giving greater effort (and playing smarter) on the defensive side of the ball.
Wouldn't this also be consistent with the conventional-wisdom thinking that coaches have a greater impact on the defensive end than on offense and that APM is crediting players for the work of their coaching staff? Quote:
In other words, using multiple seasons' worth of data, we can still get low-noise estimates of each player's performance in any given season, despite the fact that some players will vary quite a bit on the defensive end from year to year.
I think this response tends to understate potential aging effects in the data. For someone like, say, Michael Finley, including numbers from several seasons ago when he was still relatively in his prime doesn't seem to help us understand his ability at this point in his career.

Page 3

Author Message bchaikin



Joined: 27 Jan 2005
Posts: 689
Location: cleveland, ohio
Posted: Sun Feb 15, 2009 2:05 pm Post subject:

"...I don't understand why you're having so much trouble understanding the concept behind on/off +/- metrics... It's really not a difficult concept to grasp..." the concept of on/off +/- is quite simple, it's some of the results of the adjusted +/- that i question... if a rating of a metric for just a few players out of the enitre league is off because of some quirk of the metric (any metric, not just adjusted +/-), that's understandable. there are going to be outliers in any system. but in this case the authors of adjusted +/- are vehement in their characterization of chris paul being a liability on defense in 07-08, despite considerable evidence to the contrary... are you also of the opinion that chris paul was a liability on defense in 07-08, despite what actual evidence (steals) and inferred evidence (hornets opposing PG eFG% when paul did/did not play) shows and what the coaches and media claim?...
Back to top

Ilardi



Joined: 15 May 2008
Posts: 265
Location: Lawrence, KS
Posted: Sun Feb 15, 2009 2:24 pm Post subject:

bchaikin wrote:
...this piece of information strongly suggests that Chris Paul is playing far better defensively. to whom? to you? the hornets team defense gets a bit better (not insignificant, but not significantly better) when paul was on the floor from one season to the next (107.6 to 106.1 pts/poss allowed), but because of something that has absolutely nothing to do with chris paul himself, and based on a sample size of data that is just 1/3 the size of the data that when paul played and is thus much more apt to be skewed, adjusted +/- comes to the conclusion that paul "...was actually a liability on defense..." one season (07-08 ) but is "...far better defensively..." the next (08-09)?... this despite the fact that the defensive pts/poss allowed when he actually did play changed little? and despite the fact that the 07-08 season that adjusted +/- finds paul to be a liability on defense there are many others who have found his defense to be quite good? after all he received not only the most votes among all PGs for the all-D team, but also the 7th most votes among all players in the entire league for DPOY? and this from the people that have actually seen him play alot - how much did you actually see chris paul play last year?... tell us then - knowing that the number of steals paul got both seasons is about the same (and each is a defensive stop), and the data for eFG% allowed seems to show at most 1 additional defensive stop through missed FGAs forced every 2 games, just what exactly has chris paul - himself - done this year that he did not do last year such that he is "...playing far better defensively..."?... do you have any evidence whatsoever that paul has forced significantly more non-steal turnovers, or that he was personally responsible for forcing more missed FGAs by the players guarded by his teammates (that might not be reflected in data such as that found on 82games.com) this year than last year?... "...a substantial drop in the effectiveness of Paul's opposing PGs in 08-09. Bchaikin attempts to dismiss this latter point, waving away, for example, the impact of the drop in eFG% - down to 46.7% from last year's 51.8% - by calculating a net of roughly 36 more defensive stops per season, but he ignores the fact that the bulk of a player's defensive efficiency lies in his contribution to team defense - rotations, double-teams, transition coverage, traps, etc.; if a player's on-ball defense is improved, presumably it's not a stretch to infer that his off-ball defense has experienced commensurate improvements, as well;..." oh, i get it. let's just completely ignore the actual data that does in fact exist, and then infer what we will to validate our conclusions. i'll ask again, how much have you actually seen chris pual play? because in the opinions of those who have seen him play the most (the coaches and media) he was a very good defender in 07-08... "...bchaikin might also ponder what Paul's opposing PGs would have yielded in collective FG% this season had the Hornets benefited from the 07-08 version of Chandler...." oh, so now not only something that happened when chris paul was not on the floor, but something that has not happened period, is futher validation of adjusted +/- conclusions?...
Bob, I've already explained, at some length, several converging lines of evidence that support the validity of Paul's defensive APM improvement from last season to this. Given the increasingly sarcastic, snarky tone of your posts, I hope you'll understand if I fail to repeat the exercise or to respond further. Time is precious.
Back to top

Neil Paine



Joined: 13 Oct 2005
Posts: 774
Location: Atlanta, GA
Posted: Sun Feb 15, 2009 2:25 pm Post subject:

bchaikin wrote:
are you also of the opinion that chris paul was a liability on defense in 07-08, despite what actual evidence (steals) and inferred evidence (hornets opposing PG eFG% when paul did/did not play) shows and what the coaches and media claim?...
I wouldn't have used the term "liability", no... That implies that somehow his defense is bad enough to offset his offense, which would basically be impossible given his individual box score stats, his high % of team minutes and NOH's great point differential. Was he overrated because of the steals, though? Probably. I distrust steals as a primary defensive metric on principle -- you can pad them by gambling at the team's expense. CP3 is far from the worst offender at this, but it's always a concern. Subjectively, I'd say he was somewhere above average for PGs, maybe not 2nd-team All-Defense caliber, but he's got good strength, quickness, a high basketball I.Q., though his size is a liability. I have no numbers to back that up, just what I've seen of him play in college and the pros.
Back to top

Ilardi



Joined: 15 May 2008
Posts: 265
Location: Lawrence, KS
Posted: Sun Feb 15, 2009 2:34 pm Post subject:

davis21wylie2121 wrote:
Ilardi wrote:
My primary goal in APM analyses is to provide a metric of a player's actual contribution to the team's bottom line in a given season, not to provide an indicator of a player's potential contribution (e.g., if he were to start playing harder, adopt a different role in a different system, etc.) ... as Bastillon implies, it would be a mistake to look at a player's defensive APM number from a given season and conclude: this is a valid rating of Player X's defensive ability. Rather, it's merely an indicator of his defensive contribution over the period in question.
Personally, I like APM, and I believe there's a lot of potential there as APBRmetrics moves into the future. But I guess the question a lot of people would have, given what I quoted, is "what's the point of APM?" I mean, if I'm a GM, I'm probably not asking how good a player was in the past, I'm asking how good he is now -- and more importantly, what can I expect out of him in the future? This is one of those "value vs. ability" questions we always run into, and I've found that while "ability" is the real practical concern for front offices, "value" is one of those almost purely academic exercises regarding past players that's fun for discussion, but of little use to decision-makers in the here and now (besides, in APM's case, it can't even describe the past value of players before PBP was widely available). So I suppose the question is, if APM doesn't really try to predict future performance, and it can't tell us about past performance except over the past handful of seasons, doesn't that kind of limit its utility? (Just playing devil's advocate here, btw)
Davis, Thanks for the interesting questions. Of course, I think you could ask them of any metric: they all describe what has happened, not what will happen. That being said, despite the moderate degree of year-to-year fluctuation in defensive APM, there's still a great deal of stability on balance (e.g., KG is still a highly-rated player this year on defensive APM, as are Camby, Chuck Hayes, Yao, Kirilenko, Aldridge, etc.). In a nutshell (as we psychologists like to say): the best predictor of future behavior is past behavior. A corollary: the best predictor of future offensive/defensive APM is past APM. Thus, knowledge of a player's recent offensive and defensive APM ratings should be highly useful to front-office decision-makers. But they'll want to keep in mind that there's probably lots of defensive upside for poorly rated defensive APM players if they happen to be highly athletic and possessed of high basketball iq's (and are lucky enough to play for a coaching staff that can motivate them).Last edited by Ilardi on Sun Feb 15, 2009 2:46 pm; edited 1 time in total
Back to top


Last edited by Crow on Thu May 12, 2011 3:13 am, edited 1 time in total.

Top
 Profile  
 
PostPosted: Fri Apr 15, 2011 5:47 pm 
Offline

Joined: Thu Apr 14, 2011 11:10 pm
Posts: 2548
Ilardi



Kevin Pelton wrote:
Ilardi wrote:
What I'm discovering is that offensive APM (which correlates quite heavily with traditional boxscore stats, and metrics like PER that are derived from them) is more stable from year to year than is defensive APM. This isn't terribly surprising, but I think it's an important discovery with myriad implications. It suggests, for example, that many players have the potential to greatly increase their on-court impact by giving greater effort (and playing smarter) on the defensive side of the ball.
Wouldn't this also be consistent with the conventional-wisdom thinking that coaches have a greater impact on the defensive end than on offense and that APM is crediting players for the work of their coaching staff? Quote:
In other words, using multiple seasons' worth of data, we can still get low-noise estimates of each player's performance in any given season, despite the fact that some players will vary quite a bit on the defensive end from year to year.
I think this response tends to understate potential aging effects in the data. For someone like, say, Michael Finley, including numbers from several seasons ago when he was still relatively in his prime doesn't seem to help us understand his ability at this point in his career.
Kevin, Yes, I agree that the best coaches/staffs are the ones able to motivate their players to greater effort on the defensive end, and thus - as you suggest - player defensive APM ratings likely reflect (to an unknown extent, but one that's non-trivial) the impact of coaching. It's ultimately an empirical question, and one I hope to address in future models that include variables for the effect of coaching staffs (above and beyond player effects). I also agree with your basic point about "maturation effects" (declining performance in later years, improving performance in early years), over time in multi-year datasets. However, given that prior seasons are weighted much less heavily than the target (i.e., current) season in any multi-year APM model, the net biasing effect of such maturation processes should be modest. Nevertheless, I've toyed with the idea of using time-trend analyses (i.e., latent growth curve models) to model each player's APM trajectory over time, which should help smooth out some of the year-to-year noise and (hopefully) eliminate some of the bias that you've identified.
Back to top

Neil Paine



Joined: 13 Oct 2005
Posts: 774
Location: Atlanta, GA
Posted: Sun Feb 15, 2009 2:45 pm Post subject:

Thanks. I suppose my main concern was along the lines of Kevin's earlier post about aging -- maybe you reduce the standard error by including as many past seasons as possible, but how much does the inclusion of somebody's 2003 season help us understand how good he is now? I can see a lot of instances where it might actually make the estimate of his "true ability" worse, especially for older players. On a related note, are the weights of past seasons in the multiyear APM geared to produce the lowest standard errors, or to best predict next year's APM? Or is there a difference?
Back to top

deepak



Joined: 26 Apr 2006
Posts: 665
Posted: Sun Feb 15, 2009 3:17 pm Post subject:

My intuition tells me that the truth about a player, or team, usually lies somewhere between the average and what the past numbers tell us. So, if a team wins 60% of its games half-way through the year, they'll probably win between 50% and 60% the rest of the way. Let's suppose that idea applies here (it may not, not sure), and we have a player who rates very strongly defensively with, say, a +8 per 100 possessions. Then wouldn't it be more likely his actual defensive contribution is less than +8 instead of greater than +8? If that thinking is correct, than to really get at the overall contributions of a player based on APM, we should probably weight his offense more than his defense because offense is less volatile (and, so, less likely to tend to the middle). So, if a player is a -8 on offense and a +8 on defense, he's more likely to be a net negative than a net positive. Does that make sense?
Back to top

bchaikin



Joined: 27 Jan 2005
Posts: 689
Location: cleveland, ohio
Posted: Sun Feb 15, 2009 3:51 pm Post subject:

I wouldn't have used the term "liability", no... That implies that somehow his defense is bad enough to offset his offense, which would basically be impossible given his individual box score stats, his high % of team minutes and NOH's great point differential. exactly, and that is what a few others are failing to see... let's take a look at the on/off +/- for chauncey billups. the nuggets on/off pts/poss allowed on defense with/without billups this year is 108.0/103.4, and billups has played 2/3 of the team's minutes at PG and leads the team in minutes played. are we to come to the conclusion that billups is not a good defender based on this? despite the fact that denver as a team has the league's 6th best/lowest defensive pts/poss allowed? i sure wouldn't... also take a look at detroit's on/off pts/poss allowed on defense with billups for 06-07 and 07-08 when the pistons were 7th and 4th best in the league in defense. its 105.9/103.3 for 06-07 and 104.3/103.2 for 07-08, with billups again playing close to 2/3 of the team's minutes at PG. so for not 1 or 2 but for 3 straight seasons the teams that chauncey billups has played for have been better on defense without him than with him. are we to come to the conclusion that he has been a poor defender based on this data? i sure wouldn't... if next season the nuggets are much worse on defense for the 1/3 to 1/4 of the time billups does not play, but similar or slightly better with billups, yet the team defense overall is about the same as this year (like chris paul and the hornets in 07-08 and 08-09), should we then come to the conclusion that billups was a much worse defender in 08-09 than in 09-10? again i sure wouldn't... I've already explained, at some length, several converging lines of evidence that support the validity of Paul's defensive APM improvement from last season to this. and conveniently ignored evidence that is contrary to your conclusion... for example paul's high steals. last year he got 217 and this season he is on pace for a similar number. a steal is a defensive stop, a blocked shot is not. a blocked shot has to be rebounded for it to be a stop. let's say for example that about 60% of all blocked shots are rebounded by the defense. so 217/.60 is about 360 blocked shots that would be needed to get as many defensive stops as 217 steals. yet you are calling paul a liability on defense one season and a heckuva defender the next, with the same amount of steals. are there any shot blockers who have gotten as many as 350 blocked shots two seasons in a row (heck even just 300) that you that would consider were a liability on defense one season but a "...far better..." defender the next?... the fact is that statistically speaking you are grossly undervalueing the impact of paul's high steals... Given the increasingly sarcastic, snarky tone of your posts... sarcastic? because you refuse to debate the evidence contrary to your findings? snarky? because i ask you if you have actually seen chris paul play, when those who actually saw him play on a consistent basis in 07-08 (coaches/media) voted that he was a top defender when your methodology presents results at the opposite end of the spectrum?... did you or did you not open this thread with this comment: Chris Paul's defensive rating is dramatically higher than it was last year (improving from -4.54 to +6.78 this season)... I'd be curious to hear your thoughts on what (if anything) might account for this marked improvement, especially from those of you who have watched the Hornets extensively for the past couple seasons... if you are looking for open dialogue as to the perceived pros and cons of your adjusted +/- methodology based on its findings that you have presented, then this is the place. if however you are only after praise and accolades, then... I hope you'll understand if I fail to repeat the exercise or to respond further. i understand...
Back to top

Ilardi



Joined: 15 May 2008
Posts: 265
Location: Lawrence, KS
Posted: Sun Feb 15, 2009 8:20 pm Post subject:

davis21wylie2121 wrote:
Thanks. I suppose my main concern was along the lines of Kevin's earlier post about aging -- maybe you reduce the standard error by including as many past seasons as possible, but how much does the inclusion of somebody's 2003 season help us understand how good he is now? I can see a lot of instances where it might actually make the estimate of his "true ability" worse, especially for older players. On a related note, are the weights of past seasons in the multiyear APM geared to produce the lowest standard errors, or to best predict next year's APM? Or is there a difference?
Great questions. Whenever we derive APM estimates, there's going to be an interesting trade-off: on the one hand, we always want estimation errors to be as low as possible, while on the other hand we typically want to know how each player is performing right now. If the only issue were minimizing errors, we would include as many seasons as possible, and give them all equal weighting (differential weighting across seasons actually adds non-trivial noise). But if we care about how each player is doing this season (or next), we clearly don't want to give equal weighting to past seasons. And yet if we drop all past seasons entirely from the model, error terms get so large that the resulting estimates are not terribly informative. So, I've experimented with many different approaches, but all involve including prior seasons for noise reduction, but giving them such low weightings (compared with the present season) that they don't introduce enormous bias into the estimates for players that were much better or much worse a few years ago. Still, I think the next step will involve generating a multi-year APM profile for each player - separate offense/defense APM estimates for each season over a 6+ year span - and then projecting each player's ensuing "growth curve" forward (adjusting for age, position, experience, etc.) in order to more accurately project the following season's performance. I guess that'll be the next article you see from me on 82games . . .Last edited by Ilardi on Mon Feb 16, 2009 12:04 am; edited 1 time in total
Back to top

Ilardi



Joined: 15 May 2008
Posts: 265
Location: Lawrence, KS
Posted: Sun Feb 15, 2009 10:26 pm Post subject:

deepak_e wrote:
My intuition tells me that the truth about a player, or team, usually lies somewhere between the average and what the past numbers tell us. So, if a team wins 60% of its games half-way through the year, they'll probably win between 50% and 60% the rest of the way. Let's suppose that idea applies here (it may not, not sure), and we have a player who rates very strongly defensively with, say, a +8 per 100 possessions. Then wouldn't it be more likely his actual defensive contribution is less than +8 instead of greater than +8? If that thinking is correct, than to really get at the overall contributions of a player based on APM, we should probably weight his offense more than his defense because offense is less volatile (and, so, less likely to tend to the middle). So, if a player is a -8 on offense and a +8 on defense, he's more likely to be a net negative than a net positive. Does that make sense?
You're referring to an effect often referred to among statisticians as "regression to the mean" - i.e., the fact that outlier (high/low) measurements typically move closer to the mean when re-assessed. Ultimately, the degree to which such regression-to-mean effects occur on offensive vs. defensive APM estimates is an interesting empirical question (one readily capable of being addressed), but I suspect you're probably on to something with the suggestion that the regression will be greater for outlier defensive vs. offensive APM values.
Back to top

cherokee_ACB



Joined: 22 Mar 2006
Posts: 157
Posted: Mon Feb 16, 2009 10:01 am Post subject:

Ilardi wrote:
If the only issue were minimizing errors, we would include as many seasons as possible, and give them all equal weighting (differential weighting across seasons actually adds non-trivial noise).
The noise in adjusted ratings have, in my view, two components: an error per-position and team, and an individual noise component. Including multiple seasons mainly reduces the first one, but has little impact on the individual component. Case in point: Paul and Pargo. Let's compare their single season (1S) adjusted ratings with their ratings derived from multiple seasons (MS) Code:
Paul Pargo Diff Off 1S 10.8 0.3 +10.5 Off MS 9.2 -1.1 +10.3 Def 1S -10.4 -4.3 -6.1 Def MS -4.5 0.5 -5.0
When Steve and Aaron added several seasons, it moved the ratings of both players up & down, but the difference remained stable. The single season correlation is not much sure about the defensive strength of New Orleans PGs, but it can tell pretty well how much better Pargo was. What can we do about this? For once, we can try to separate the two noise components, by deriving the individual noise contribution using probability theory. Or we can tell the correlation to attempt to find the difference between starters and bench players of a team, instead of the rating for each player. We'd compute the coefficients for Paul and for (Paul-Pargo), which should give a good estimate (i.e. with a low error) of the adjusted difference between both players.
Back to top

Ilardi



Joined: 15 May 2008
Posts: 265
Location: Lawrence, KS
Posted: Mon Feb 16, 2009 12:48 pm Post subject:

cherokee_ACB wrote:
Ilardi wrote:
If the only issue were minimizing errors, we would include as many seasons as possible, and give them all equal weighting (differential weighting across seasons actually adds non-trivial noise).
The noise in adjusted ratings have, in my view, two components: an error per-position and team, and an individual noise component. Including multiple seasons mainly reduces the first one, but has little impact on the individual component. Case in point: Paul and Pargo. Let's compare their single season (1S) adjusted ratings with their ratings derived from multiple seasons (MS) Code:
Paul Pargo Diff Off 1S 10.8 0.3 +10.5 Off MS 9.2 -1.1 +10.3 Def 1S -10.4 -4.3 -6.1 Def MS -4.5 0.5 -5.0
When Steve and Aaron added several seasons, it moved the ratings of both players up & down, but the difference remained stable. The single season correlation is not much sure about the defensive strength of New Orleans PGs, but it can tell pretty well how much better Pargo was. What can we do about this? For once, we can try to separate the two noise components, by deriving the individual noise contribution using probability theory. Or we can tell the correlation to attempt to find the difference between starters and bench players of a team, instead of the rating for each player. We'd compute the coefficients for Paul and for (Paul-Pargo), which should give a good estimate (i.e. with a low error) of the adjusted difference between both players.
Thanks, Cherokee - that's an intriguing thought. I think I'd be inclined to try teasing apart these different sources of noise in an HLM (hierarchical linear model), using team/position as superordinate grouping variables and player as lower-level variables. The problem is that such models can get really messy and unwieldy, and they don't always reduce noise the way one would hope. Maybe you've got some thoughts on a better way of going about it (Bayesian estimation, etc.)? By the way, I'm not convinced that the Paul/Pargo example you provided is completely representative of the way APM always treats starter-backup pairs. Just eyeballing other pg pairs last year like Rondo/House, Bibby/Law, and Billups/Stuckey . . . I didn't see quite the same consistency in (starter - backup) across single-year and multi-year estimates. If you (or someone else on APBR) has time to look at this systematically, I'd be highly curious to know if it's actually a stable phenomenon.
Back to top

cherokee_ACB



Joined: 22 Mar 2006
Posts: 157
Posted: Mon Feb 16, 2009 2:20 pm Post subject:

Ilardi wrote:
By the way, I'm not convinced that the Paul/Pargo example you provided is completely representative of the way APM always treats starter-backup pairs. Just eyeballing other pg pairs last year like Rondo/House, Bibby/Law, and Billups/Stuckey . . . I didn't see quite the same consistency in (starter - backup) across single-year and multi-year estimates.
Of course, I choose a convenient example, but there are others. For Rondo/House and Billups/Stuckey playoff data did have an impact which Eli's numbers didn't catch (this can be seen in bv.com ratings pre and post playoffs); for Bibby/Law I don't have a quick explanation. Anyway, in all cases the individual ratings tend to move more than the rating difference. Regarding hierarchical models, I'm not familiar with them. Bayesian estimation could be useful to deal with reference players, or as a way to mix statistical and adjusted +/-, but I haven't given many thoughts to it.
Back to top

bastillon



Joined: 04 Nov 2008
Posts: 55
Posted: Mon Feb 16, 2009 3:33 pm Post subject:

bchaikin, 1. Mentioning players/coaches opinions without their arguements makes no sense. It's not important what they think, but why. They didn't give any reason. For example Kobe Bryant is allegedly, according to coaches/players nominations/awards the all-NBA caliber defender through the whole last decade. Even though some of his seasons(especially after Shaq has left him) he didn't really bother by defending opposing players or giving any effort whatsoever so he could save his energy for the Lakers' poor-at-the-time offense. Some awards are given for the hype or names. We have to be careful judging by that. League tend to overrate players for some reason - maybe it's the numbers, maybe names, maybe conditions, anyway it's a fact and there are many examples you certainly know. We can just look at the Hoopshype's salaries and see how many overrated players there have been in the NBA. So my point is that it doesn't matter who says, but what says. 2. Players recording high number of steals are often overrated. That's exactly what happened to Paul last season. There's no way he could be 7th best defender in the league. Not even close. First off, small guys don't have that kind of influence on team defense that bigs have. Not surprising it's big who are leading in defensive categories and also having much more success getting awards. It happens hardly ever that elite guard could have impact on team defense good big man has. If you have the team with dominant big men inside and poor defenders at 1-3 positions, your team defense could get away with it, because even if your small guys are matador-type players, there's always that 2nd line of defense. Second part is rebounding, you can have Kidds, Rondos all you want, but without rebounding bigs your team is weak on the boards anyway - just look at the Kidd's-era Nets. Given that defending the paint and rebounding are two of the crucial elements of team defense, small guys have much less effect on it(stats also confirm it). If so, Paul not only must have been one of the elite guard defenders(which he wasn't) he must have been better than most of the good defensive big guys. Would you really make a case that Paul was better than Garnett, Duncan, Camby, Okafor, Howard, Yao, Sheed ? I don't think you would. Being a realist, Paul wasn't even close to deserve 7th place on DPOTY list and even him being a candidate is pretty ridiculous. That's about theory... 3. You're making good points about the evidence but you don't take into account stats not showed in the boxscore. You count the steal as a defensive stop as if gambling wasn't causing total defensive breakdowns. We can only suppose how many steals per failed gamblings Paul has had, but it's sure he had some. Only then we can trust your evidence based on steals area. Blocks maybe don't count as a defensive stops, but they intimidate from attacking the paint. Shot blocker over gambling little guy stealing the ball any day of the week. Besides, it's not only missed shots and turnover player can cause. Being a defender you care about staying in front of the opponent first, not to draw help from your teammates, not letting the defense collapse. Then there is pick and roll defense, which is particularly important as for as PGs concerned. Then there are rotations - how quick you are, how hard you contest. You simply eliminate many of the important factors to team defense in order to make your points look more legititimate. On purpose or not, it's not fair.
Back to top

gabefarkas



Joined: 31 Dec 2004
Posts: 1313
Location: Durham, NC
Posted: Mon Feb 16, 2009 4:23 pm Post subject:

davis21wylie2121 wrote:
bchaikin wrote:
the hornets team defense gets a bit better (not insignificant, but not significantly better) when paul was on the floor from one season to the next (107.6 to 106.1 pts/poss allowed), but because of something that has absolutely nothing to do with chris paul himself, and based on a sample size of data that is just 1/3 the size of the data that when paul played and is thus much more apt to be skewed, adjusted +/- comes to the conclusion that paul "...was actually a liability on defense..." one season (07-08 ) but is "...far better defensively..." the next (08-09)?...
At the same time, I don't understand why you're having so much trouble understanding the concept behind on/off +/- metrics, Bob... When a player is on the court, the team's DRtg is not only a function of his ability, but that of 4 other guys. So how do you isolate his specific contribution? You find instances in which some combination of the other 4 guys are on the court, and the player we're studying isn't. Is the team's defense better? That suggests the other 4 guys were contributing more to the team's DRtg when our guy was on the court (which is essentially what happened in '08: NOH was 6.2 efficiency points worse on D when Paul was in the game vs. when he wasn't). Or is the defense a lot worse? That suggests our guy was actually making a far bigger contribution than the other 4 guys when he was on the floor (like Paul in '09: NOH is 5.7 DRtg points better when he's on vs. off). It's really not a difficult concept to grasp on paper -- the hard part is getting around the fact that players often play with generally the same group of players, because it's tough to isolate one specific player vs. the other 4 guys on the court with him when he's always playing with the same 4 guys, and they're always sitting when he's sitting.
Part of the problem lies in the fact that it's not really a true plus/minus statistic like raw +/- is. Raw +/- measures the difference on both ends of the court. Once you start separating it out into "offensive" and "defensive" components, it ceases to be an interpretable +/- and instead it becomes just a "plus" (Offensive) and a "minus" (Defensive), separate from each other.


Page 4



Author Message gabefarkas



Joined: 31 Dec 2004
Posts: 1313
Location: Durham, NC
Posted: Mon Feb 16, 2009 4:29 pm Post subject:

Ilardi wrote:
Of course, I think you could ask them of any metric: they all describe what has happened, not what will happen. That being said, despite the moderate degree of year-to-year fluctuation in defensive APM, there's still a great deal of stability on balance (e.g., KG is still a highly-rated player this year on defensive APM, as are Camby, Chuck Hayes, Yao, Kirilenko, Aldridge, etc.).
Dr Ilardi - can you explain what the part in bold means? To me, "stability" and "balance" are essentially synonyms, no?
Back to top

Ilardi



Joined: 15 May 2008
Posts: 262
Location: Lawrence, KS
Posted: Mon Feb 16, 2009 4:53 pm Post subject:

gabefarkas wrote:
Ilardi wrote:
Of course, I think you could ask them of any metric: they all describe what has happened, not what will happen. That being said, despite the moderate degree of year-to-year fluctuation in defensive APM, there's still a great deal of stability on balance (e.g., KG is still a highly-rated player this year on defensive APM, as are Camby, Chuck Hayes, Yao, Kirilenko, Aldridge, etc.).
Dr Ilardi - can you explain what the part in bold means? To me, "stability" and "balance" are essentially synonyms, no?
By stability I mean "temporal stability" - i.e., the value of the rating tends to be fairly similar each time it's measured across multiple time points (e.g., from one season to the next). By on balance I simply mean "all things considered" (i.e., as everything is weighed in the balance). P.S. Please feel free to call me Steve in this forum (unless you happen to be one of my undergrads here at KU).
Back to top

bchaikin



Joined: 27 Jan 2005
Posts: 685
Location: cleveland, ohio
Posted: Mon Feb 16, 2009 11:30 pm Post subject:

Mentioning players/coaches opinions without their arguments makes no sense. It's not important what they think... perhaps you should read some of the books of bill james. he was of the belief that the opinions of those who saw players play the most (sportswriters and coaches) was of great import, to be used in conjunction with stats analysis, although admittedly he was speaking about baseball not basketball, and in a historical context... For example Kobe Bryant is allegedly, according to coaches/players nominations/awards the all-NBA caliber defender through the whole last decade. Even though some of his seasons(especially after Shaq has left him) he didn't really bother defending opposing players or giving any effort whatsoever so he could save his energy for the Lakers' poor-at-the-time offense. this by chance the same kobe bryant that's been named to the all-D team 8 times in his career (6 all-D 1st team, 2 all-D 2nd team)? just out of curiosity, are there any other players voted to the all-D team, even 1/2 the amount of times bryant was, that in your opinon didn't defend opposing players or give any effort on defense whatsoever?... Players recording high number of steals are often overrated. That's exactly what happened to Paul last season. often, huh?... ok... chris paul is on pace to get 200+ steals in a season for the 2nd time. here is a list of the nba players that have gotten 200+ steals in a season more than once in their careers over the past 30 seasons: mookie blaylock maurice cheeks clyde drexler rickey green eddie jordan michael jordan fat lever gary payton scottie pippen michael ray richardson alvin robertson john stockton if, as you say, high steal players are often overrated as defenders, would you care to share your expertise with this discussion group as to which of these players were not very good defenders, especially in the years they got their 200+ steals?... There's no way he could be 7th best defender in the league. Not even close. no? you think that's too low? maybe 6th, 5th, or 4th best?... First off, small guys don't have that kind of influence on team defense that bigs have. from 1993-94 to 1996-97, for a 4 year stretch, the atlanta hawks had the league's 5th best/lowest pts/poss allowed on defense in the league. only new york, chicago, seattle, and portland had better defenses... that atlanta team those 4 years has just the 11th best/lowest eFG% allowed on defense, just the 15th highest BS/g, and had a defensive rebounding percentage of available rebounds that was just 18th in the league. yet they had the league's 5th best defense... a key reason why was that over these 4 years the hawks averaged the 2nd most steals per game among all teams (9.5 ST/g)... these 4 seasons their 6-1 and 185 lb PG mookie blaylock got more steals than any other player in the league, 836 steals. he averaged 209 steals a year... only once in these 4 years did a hawks player block more than 150 shots in a season (mutombo, 96-97, his only season with the hawks in this 4 year stretch), and only two other times did a player block even 100 shots (andrew lang and jon konchak)... blaylock played over 3600 more total minutes than any other hawks player these 4 years, was named to the all-D team all 4 seasons (twice 1st, twice 2nd team), and the only other hawks player to be named to the all-D team during this time was mutombo in 96-97.... i'd say small guy mookie blaylock had quite an influence on one of the league's better defensive teams over a good 4 year stretch... Second part is rebounding, you can have Kidds, Rondos all you want, but without rebounding bigs your team is weak on the boards anyway - just look at the Kidd's-era Nets. Given that defending the paint and rebounding are two of the crucial elements of team defense, small guys have much less effect on it(stats also confirm it). you mean the 01-02 and 02-03 new jersey nets team that jason kidd led to the finals two seasons in a row? that nets team was the league's best team defensively (lowest pts/poss allowed on defense) in the league those two years combined. yet that team for those two seasons had just the 9th best rate of defensive rebounding percentage of available rebounds, just the 7th best rate of defensive rebounds per game, and just the 13th best rate of BS/g... but guess what? they were 5th best in steals per game, and guess who got the 2nd most total steals of all players in the league these two seasons? yep, jason kidd, with 354 steals - which was over 120 more steals than what the next best nets player got (kittles with 231)... Being a realist, Paul wasn't even close to deserve 7th place on DPOTY list and even him being a candidate is pretty ridiculous. evidently those who voted for the DPOY think differently than you. perhaps they aren't as real as you... You count the steal as a defensive stop as if gambling wasn't causing total defensive breakdowns. We can only suppose how many steals per failed gamblings Paul has had, but it's sure he had some. what about all those times paul went for a steal, didn't get it, but forced a turnover or a bad pass? or are you under the impression that every single time a player goes for a steal but doesn't get it it's bad for his team?... Being a defender you care about staying in front of the opponent first, not to draw help from your teammates, not letting the defense collapse. being a defender you care about stopping your opponent from scoring, first and foremost. getting a steal not only stops your opponent from scoring, it stops him from shooting in the first place... You simply eliminate many of the important factors to team defense in order to make your points look more legitimate. On purpose or not... oh it's on purpose... ...it's not fair. all's fair in love and apbrmetrics...
Back to top

bastillon



Joined: 04 Nov 2008
Posts: 55
Posted: Tue Feb 17, 2009 1:02 am Post subject:

I'm just saying that your steal numbers aren't worth too much, if you don't know how many times the defense suffered from his gambling. As for small vs big conversation I thought it would be obvious... Look at the awards, look at the teams without interior defense, the DRtg, APM +/-. It's all evidence. Really surprised me that you're even questioning bigs impact.
Back to top

haralabob



Joined: 11 Apr 2007
Posts: 25
Posted: Tue Feb 17, 2009 1:40 pm Post subject:

Although I enjoy reading the back and forth and heated debates, I couldn't help myself on this subject. Adjusted plus minus and defensive counterpart statistics shed a little light on who is or isn't a good defensive player. But obviously they don't tell the whole story, if you want to know how good Chris Paul was on defense in 2007-08 chart every single NOR defensive possession. Here are some numbers; The guy he was guarding scored .87 points per offensive action, (on 1203 actions). The best number on his team was .80 held by Rasual Butler (313 actions) Pargo's numbers were 1.01 on 620 actions.
Back to top

gabefarkas



Joined: 31 Dec 2004
Posts: 1313
Location: Durham, NC
Posted: Tue Feb 17, 2009 2:07 pm Post subject:

Ilardi wrote:
gabefarkas wrote:
Ilardi wrote:
Of course, I think you could ask them of any metric: they all describe what has happened, not what will happen. That being said, despite the moderate degree of year-to-year fluctuation in defensive APM, there's still a great deal of stability on balance (e.g., KG is still a highly-rated player this year on defensive APM, as are Camby, Chuck Hayes, Yao, Kirilenko, Aldridge, etc.).
Dr Ilardi - can you explain what the part in bold means? To me, "stability" and "balance" are essentially synonyms, no?
By stability I mean "temporal stability" - i.e., the value of the rating tends to be fairly similar each time it's measured across multiple time points (e.g., from one season to the next). By on balance I simply mean "all things considered" (i.e., as everything is weighed in the balance). P.S. Please feel free to call me Steve in this forum (unless you happen to be one of my undergrads here at KU).
In that case, it seems the issue with Chris Paul is even more exaggerated, as he went from one extreme to the other, no?
Back to top

Ilardi



Joined: 15 May 2008
Posts: 262
Location: Lawrence, KS
Posted: Tue Feb 17, 2009 2:35 pm Post subject:

haralabob wrote:
Although I enjoy reading the back and forth and heated debates, I couldn't help myself on this subject. Adjusted plus minus and defensive counterpart statistics shed a little light on who is or isn't a good defensive player. But obviously they don't tell the whole story, if you want to know how good Chris Paul was on defense in 2007-08 chart every single NOR defensive possession. Here are some numbers; The guy he was guarding scored .87 points per offensive action, (on 1203 actions). The best number on his team was .80 held by Rasual Butler (313 actions) Pargo's numbers were 1.01 on 620 actions.
Those are fascinating numbers, but I have a few questions (if you don't mind) that might help me with their interpretation: 1) what does and does not count as an "offensive action" (i.e., how is the term operationalized)? 2) is your points-per-offensive-action stat normed by position? (presumably, some positions on offense engage in a higher proportion of non-scoring actions [e.g., setting screens, handoffs to regain dribble, "hockey assists, etc,"] than others) 3) what counts as the "player being guarded"? if a pg picks up a pf on a screen, is he now guarding him for the purposes of your stat? what if he tries to cover the pf's man in transition because the pf didn't get back? what if he's part of a double-team of someone else's man? or a zone? or a trap? 4) how did Paul's .87 ppoa in 07-08 rank among other 1's in the league? 5) how did the points-per-offensive-action rating of Paul's teammates vary when he was on-versus-off the court? Since Paul's overall adjusted plus-minus in 07-08 was negative, it would seem likely that his teammates' defensive ppoa numbers must have increased when he was on the court (assuming Paul's own .87 number is regarded as quite low)? Many thanks in advance for any clarification you can provide. P.S. Where can the rest of us get ahold of your points-per-action numbers?Last edited by Ilardi on Tue Feb 17, 2009 2:40 pm; edited 1 time in total
Back to top

Ilardi



Joined: 15 May 2008
Posts: 262
Location: Lawrence, KS
Posted: Tue Feb 17, 2009 2:38 pm Post subject:

gabefarkas wrote:
Ilardi wrote:
gabefarkas wrote:
Ilardi wrote:
Of course, I think you could ask them of any metric: they all describe what has happened, not what will happen. That being said, despite the moderate degree of year-to-year fluctuation in defensive APM, there's still a great deal of stability on balance (e.g., KG is still a highly-rated player this year on defensive APM, as are Camby, Chuck Hayes, Yao, Kirilenko, Aldridge, etc.).
Dr Ilardi - can you explain what the part in bold means? To me, "stability" and "balance" are essentially synonyms, no?
By stability I mean "temporal stability" - i.e., the value of the rating tends to be fairly similar each time it's measured across multiple time points (e.g., from one season to the next). By on balance I simply mean "all things considered" (i.e., as everything is weighed in the balance). P.S. Please feel free to call me Steve in this forum (unless you happen to be one of my undergrads here at KU).
In that case, it seems the issue with Chris Paul is even more exaggerated, as he went from one extreme to the other, no?
No question: he's an outlier. It'll be interesting to see to what extent his high rating this season holds up through 82 games & playoffs.
Back to top

bastillon



Joined: 04 Nov 2008
Posts: 55
Posted: Tue Feb 17, 2009 3:56 pm Post subject:

haralabob wrote:
Although I enjoy reading the back and forth and heated debates, I couldn't help myself on this subject. Adjusted plus minus and defensive counterpart statistics shed a little light on who is or isn't a good defensive player. But obviously they don't tell the whole story, if you want to know how good Chris Paul was on defense in 2007-08 chart every single NOR defensive possession. Here are some numbers; The guy he was guarding scored .87 points per offensive action, (on 1203 actions). The best number on his team was .80 held by Rasual Butler (313 actions) Pargo's numbers were 1.01 on 620 actions.
It's just his man, I don't see you measured how his play affected the whole team and it's the case right here.
Back to top

haralabob



Joined: 11 Apr 2007
Posts: 25
Posted: Tue Feb 17, 2009 4:24 pm Post subject:

Hey Steve; what does and does not count as an "offensive action" (i.e., how is the term operationalized)? Roughly - We consider, shots, turnovers, and drawn shooting fouls as offensive actions - we also factor in drawn non shooting fouls in a different metric. In other words when you are actively guarding the player or engaged, if he takes a shot, turns the ball over or draws a shooting foul you are are charged as the defensive player. what counts as the "player being guarded"? if a pg picks up a pf on a screen, is he now guarding him for the purposes of your stat? what if he tries to cover the pf's man in transition because the pf didn't get back? what if he's part of a double-team of someone else's man? or a zone? or a trap? The initial numbers I gave you were just for all unfiltered possessions. If i add the following criteria; No Zone defense No Transition fastbreak defense No excessive Help Defense. The numbers look like this; Chris Paul .91 on 782 actions Rasual Butler .78 on 201 actions Jannero Pargo on 1.02 actions. Basically what we are looking at are all halfcourt offensive actions. We only remove excessive help defense that is considered above average, an example of this would be actions where Chandler or another help defender came over and blocked or severely altered the shot. how did Paul's .87 ppoa in 07-08 rank among other 1's in the league? Here is a sampling of 5 pgs filtered with the above criteria. Chauncey Billups .77 469 Jason Kidd (w/ Dallas) .81 on 349 Jameer Nelson .96 on 577 Devin Harris (w/Dallas) .98 on 277 Derek Fisher 1.00 on 670 how did the points-per-offensive-action rating of Paul's teammates vary when he was on-versus-off the court? Since Paul's overall adjusted plus-minus in 07-08 was negative, it would seem likely that his teammates' defensive ppoa numbers must have increased when he was on the court (assuming Paul's own .87 number is regarded as quite low)? At this time I am not currently interested in distributing or releasing any of the defensive data I have acquired. Haralabob
Back to top

haralabob



Joined: 11 Apr 2007
Posts: 25
Posted: Tue Feb 17, 2009 4:52 pm Post subject:

bastillon wrote:
It's just his man, I don't see you measured how his play affected the whole team and it's the case right here.
Correct, its just his man, if you wanted to know how his play impacted the rest of the team I could give you the number of times the player he was guarding got past him and was picked up by a secondary help defender. Conversely I could also give you numbers on where he came over to help on defense. Or you could come up with creative pairings to see how players X's numbers looked when A, B (and on) are on the court. There are other limitations, for instance each player you are guarding has an offensive "ev" or expected value for each action. The numbers I provided don't factor offensive 'ev' into the equation. To better understand this consider a player who spent their whole season guarding Jared Jeffries vs someone who spent their whole season guarding LeBron James. I have an adjusted defensive number that accounts for this and is more accurate. . Haralabob
Back to top

Ben



Joined: 13 Jan 2005
Posts: 264
Location: Iowa City
Posted: Tue Feb 17, 2009 4:55 pm Post subject:

haralabob wrote:
bastillon wrote:
It's just his man, I don't see you measured how his play affected the whole team and it's the case right here.
Correct, its just his man, if you wanted to know how his play impacted the rest of the team I could give you the number of times the player he was guarding got past him and was picked up by a secondary help defender. Conversely I could also give you numbers on where he came over to help on defense. Or you could come up with creative pairings to see how players X's numbers looked when A, B (and on) are on the court. There are other limitations, for instance each player you are guarding has an offensive "ev" or expected value for each action. The numbers I provided don't factor offensive 'ev' into the equation. To better understand this consider a player who spent their whole season guarding Jared Jeffries vs someone who spent their whole season guarding LeBron James. I have an adjusted defensive number that accounts for this and is more accurate. . Haralabob
So could you summarize your assessment of how Chris Paul stacks up defensively both this year and last year?
Back to top

Neil Paine



Joined: 13 Oct 2005
Posts: 774
Location: Atlanta, GA
Posted: Tue Feb 17, 2009 4:59 pm Post subject:

Ben wrote:
So could you summarize your assessment of how Chris Paul stacks up defensively both this year and last year?
Yeah, I was gonna say: Haralabos, do you think (according to your numbers) the quality of CP's defense has changed in any way between 07-08 & 08-09? Is there any way to explain his wild shift in adjusted +/- from a "bad" defender to a relatively good one?
Back to top

haralabob



Joined: 11 Apr 2007
Posts: 25
Posted: Tue Feb 17, 2009 5:11 pm Post subject:

Quote:
Yeah, I was gonna say: Haralabos, do you think (according to your numbers) the quality of CP's defense has changed in any way between 07-08 & 08-09? Is there any way to explain his wild shift in adjusted +/- from a "bad" defender to a relatively good one?
No I'd say that there isn't. The numbers I gave were for 07-08. His defensive numbers for 07-08 were above average for the PG position, his numbers for this year are also above average for the PG position.
Back to top

Neil Paine



Joined: 13 Oct 2005
Posts: 774
Location: Atlanta, GA
Posted: Tue Feb 17, 2009 5:19 pm Post subject:

haralabob wrote:
No I'd say that there isn't. The numbers I gave were for 07-08. His defensive numbers for 07-08 were above average for the PG position, his numbers for this year are also above average for the PG position.
I have to admit, this is a strange case. My eyes tell me he hasn't really done anything differently, either. I wonder, could this have simply been the "perfect storm" of small sample sizes, multicollinearity, and the Hornets' substitution patterns playing tricks on the adjusted +/- method?
Back to top


Page 5

Author Message Kevin Pelton
Site Admin


Joined: 30 Dec 2004
Posts: 979
Location: Seattle
Posted: Tue Feb 17, 2009 5:46 pm Post subject:

My subjective perspective is that we're probably spending a lot of time arguing about something that is largely noise, along the lines of what Neil suggests. Steve, could you calculate the statistical significance of the difference between Paul's defensive ratings? I don't believe you've posted error terms broken out by offense and defense for him those two seasons, plus my grasp of formal statistics is such that I would probably screw up that calculation myself with two variable distributions.
Back to top

Westy



Joined: 15 Nov 2007
Posts: 15
Location: Chicago, IL
Posted: Tue Feb 17, 2009 6:55 pm Post subject:

Quote:
haralabob wrote: "No I'd say that there isn't. The numbers I gave were for 07-08. His defensive numbers for 07-08 were above average for the PG position, his numbers for this year are also above average for the PG position." I have to admit, this is a strange case. My eyes tell me he hasn't really done anything differently, either. I wonder, could this have simply been the "perfect storm" of small sample sizes, multicollinearity, and the Hornets' substitution patterns playing tricks on the adjusted +/- method?
I guess I don't buy the 'perfect storm' proposition. If this is the APM method being 'tricked', we need to know how and why. If we can't know when to trust it, we can't trust it. If Paul hasn't changed but yet APM says he has, this would also indicate further modification/tweaking is necessary.
Back to top

mtamada



Joined: 28 Jan 2005
Posts: 377
Posted: Tue Feb 17, 2009 10:49 pm Post subject:

Westy wrote:
If Paul hasn't changed but yet APM says he has, this would also indicate further modification/tweaking is necessary.
Not necessarily. It could simply mean that (as DanR has said from the beginning) that APM has a large standard error; it is a theoretically good measure but it is not reliable, at least not when based on a single season's data. To paraphrase the closing line from the movie Chinatown: "It's statistics Jake". In small samples, all sorts of weird stuff can occur -- and WILL occur, guaranteed. It's in statistics' nature (now I'm reminded of the quote from "The Crying Game" but that's enough moive quotes.)
Back to top

Mountain



Joined: 13 Mar 2007
Posts: 1527
Posted: Wed Feb 18, 2009 4:38 am Post subject:

Using 82 games data. the opponent eFG% with Paul on the court improved this season compared to last season by 2 percentage points but the comparison of the offs showed 3 times the change to the negative. Team defensive rebounding with Paul on the court actually fell 1% but team defensive rebounding % without Paul showed 5 times the change to the negative. Year to year opponent free throw attempts with Paul are up by 5 per 48. without just 1. Turnovers are unchanged for with or without Paul. Paul is slightly better this season than last on counterpart data but given most of these factor results I have trouble seeing where he improved on help or team defense net or that much. As Bob asked before, is the off data having too much impact? Chandler's sharp fall - from injuries- from =1 to -9 but also Peja's fall from +13 to +6 and West's fall from +6 to 0 coincide with Paul's rise from =1 to +17. Playing with these same 3 teammates heavy minutes Paul has improved so much that he takes 85% of their collective credit losses? I've wondered before if adjusted overly polarizes and in this case I continue to wonder. Whatever value adjusted has, actual value derived from a player is heavily affected by the lineups he is used in. Players are not the same player all the time, they are different depending on what opportunities / competition and support or lack of necessary support (different players different types of support needed, to different degrees and different impacts) they have in those lineups. I think somebody or several somebody's (including Mark Cuban) said that a strong adjusted is a sign of a player succeeding in a role / in the lineups he was used in. I agree with that and would add that the polarizing that seems to occur on teams may make the very best role achievers shine too brightly and the rest suffer some as a consequence. So given that and errors unless you are -4 I wouldn't be that critical. Looking at Paul's five most used lineups, good for almost half his total minutes, I only find about 35 total minutes where Daniels played with the exact other 4. Not much for direct comparison of similar contexts. Paul's 6th biggest minute lineup was -7 per 46, in 34 minutes of use without Paul and +14 with. This and the overall raw Paul on/off do square with adjusted. So maybe it is on target. Hard to say. Would Daniels be better with these foursomes if he got more than scant minutes run with them? Probably. How much better? Would have to see it to know.Last edited by Mountain on Wed Feb 18, 2009 2:53 pm; edited 1 time in total
Back to top


Top
 Profile  
 
PostPosted: Fri Apr 15, 2011 5:48 pm 
Offline

Joined: Thu Apr 14, 2011 11:10 pm
Posts: 2548
Ilardi



Joined: 15 May 2008
Posts: 265
Location: Lawrence, KS
Posted: Wed Feb 18, 2009 10:12 am Post subject:

So, Chris Paul's defensive adjusted plus-minus (APM) has improved substantially from 07-08 to 08-09 (partial season), a finding that's somewhat counter-intuitive, and we're left pondering the interpretive possibilities: Maybe it's just the noisiness of the APM measure? (suggested by Kevin Pelton and others) Although the noise level in Paul's APM ratings is considerably higher than that of most other players in the league with comparable minutes (reflecting a relative lack of variability in Hornets' lineups), this cannot be the whole story. Paul's defensive APM estimate last season was -4.54 with a standard error of 1.52, so we can say with 95% confidence that his true rating last year was no better than -1.5 (since standard errors are roughly normally distributed). Conversely, his defensive APM this year (through Feb 3) is 6.76 with an se of 1.57, so we can say with 95% confidence that his true rating this year is no lower than roughly +3.5. So, despite the noise, his defensive APM rating this year is markedly improved, at a high level of statistical significance. So maybe it's an inherent flaw in the APM metric itself? Maybe, but let's review again the evidence in favor of an alternative hypothesis - that the metric is valid, and Paul is actually having a more favorable impact on defense this year: 1) During Paul's on-court minutes, the Hornets' defensive efficiency has improved from 107.6 last season to 106.1 (pts per 100 poss) this season. That improvement has occurred despite a clear deterioration in the play (and minutes) of Tyson Chandler, who was widely regarded as the anchor of the team's defense last season. 2) The team's improvement in defensive efficiency with Paul on the court this season has also occurred despite a dramatic downturn in the team's defensive performance with Paul off the court. Last year, the team was better defensively when Paul was off-court (101.4) than when he was on (107.6); this year, the situation is reversed: they're much worse defensively when he's off-court (111.8 ) than when he's on (106.1). That's a swing of over 10 point per 100 possessions in Paul's teammates' off-court performance. And remember, most of these off-court teammates also share a high percentage of their minutes with Paul on-court. (We'll come back to this point in a moment). 3) Paul's opposing PG eFG% has dropped sharply from 51.8% last year to 46.7% this year (again, without the benefit of Chandler playing in top form). Improvements are also seen this year in opposing PG turnovers, assists, iFG%, and PER. Now, it is certainly true, as Mountain and others have suggested, the APM estimates rely on the assumption that player effects on offense and defense remain reasonably constant across various lineup permutations. (This assumption is not strictly necessary, however, since the APM model can also test various player-by-player interaction effects, i.e., the possibility that the impact of Player A varies according to the presence/absence of Player B; however, no one to my knowledge has ever systematically tested such player-by-player interaction effects - a tedious, though highly worthy, task for another day). Anyway, let's consider this key assumption of "constancy of player impact", since I believe our interpretation of Paul's defensive APM improvement hinges on it. As we've seen, Paul's teammates are much worse defensively this year when Paul is off-court, while the team was actually better without Paul defensively last year. This could either reflect the fact that: (a) Paul's teammates greatly improve on defense whenever they play alongside Paul this year, whereas (for some mysterious set of reasons) they did the exact opposite last year, and got much worse on defense whenever they played alongside Paul; or (b) Paul's teammates have exerted a reasonably constant defensive effect regardless of Paul's on-court or off-court presence, but Paul has stepped up his impact on defense since last season while his teammates in aggregate (especially Chandler) have gotten worse. I happen to find the latter alternative the more parsimonious (and thus the more compelling), but I accept that tastes can certainly differ on this point. Perhaps no clear resolution will occur until I go back and test all the 2-way, 3-way, 4-way, and 5-way player interaction effects in an attempt to untangle the mystery in more systematic fashion. (Such an analysis will tell us for certain the extent to which Paul's teammates actually played better without him on d last year but better with him this year.) Since I've got lots on my plate for the next couple weeks, however, it may take until Spring Break before I'm able to carve out ample time to take this on. Stay tuned . . .
Back to top

schtevie



Joined: 18 Apr 2005
Posts: 411
Posted: Wed Feb 18, 2009 12:34 pm Post subject:

Steve, there is one particular story of player interaction, raised early on in this string, that seems to me the most compelling to investigate. And this has to do with the interaction of Chris Paul playing alongside other small guards. In 2007-08, in almost 40% of his minutes, he was running with Pargo and Jackson. (And these same guys were his teammates in '06-'07.) Chris Paul is rated by one year Def. Adj+/- as being a liability compared to these two. Perhaps it was the case that Paul, being the notionally better defender, was guarding the 2, and taking one for the team. 82games counterpart data says he didn't do even a second of that, but that can't be right. This year, according to your data, Paul has played but 9 seconds with a guard smaller than 6' 4". This might be an important part of the turnaround. Would a precise way to address this issue be to divide CP into two players: Small Paul and Large Paul, where the former would be his minutes running with small 2s and the latter when he was not at the same risk of being abused by taller opponents?
Back to top

gabefarkas



Joined: 31 Dec 2004
Posts: 1313
Location: Durham, NC
Posted: Wed Feb 18, 2009 12:45 pm Post subject:

Ilardi wrote:
gabefarkas wrote:
In that case, it seems the issue with Chris Paul is even more exaggerated, as he went from one extreme to the other, no?
No question: he's an outlier. It'll be interesting to see to what extent his high rating this season holds up through 82 games & playoffs.
I see. So you suspect it's likely that it may drop back down (towards 0) as more data is accumulated?
Back to top

Ilardi



Joined: 15 May 2008
Posts: 265
Location: Lawrence, KS
Posted: Wed Feb 18, 2009 12:46 pm Post subject:

schtevie wrote:
Steve, there is one particular story of player interaction, raised early on in this string, that seems to me the most compelling to investigate. And this has to do with the interaction of Chris Paul playing alongside other small guards. In 2007-08, in almost 40% of his minutes, he was running with Pargo and Jackson. (And these same guys were his teammates in '06-'07.) Chris Paul is rated by one year Def. Adj+/- as being a liability compared to these two. Perhaps it was the case that Paul, being the notionally better defender, was guarding the 2, and taking one for the team. 82games counterpart data says he didn't do even a second of that, but that can't be right. This year, according to your data, Paul has played but 9 seconds with a guard smaller than 6' 4". This might be an important part of the turnaround. Would a precise way to address this issue be to divide CP into two players: Small Paul and Large Paul, where the former would be his minutes running with small 2s and the latter when he was not at the same risk of being abused by taller opponents?
Thanks, Schtevie. Yes, I think the potential interaction effect you've raised is a promising hypothesis (originally raised by Mountain, I think), and certainly worth testing, though I think rather than creating 2 separate CP variables, I'd be more inclined to directly test CP*Pargo and CP*Jackson interaction effects. Such interactions (i.e., the hypothesized deleterious impact of CP3 on defense when playing in tandem with other short-ish 1s) may indeed explain a lot; on the other hand, they may not. But it's ultimately an empirical question, and - fortunately - one that can be addressed with actual data. Please note, however, that even if this interaction effect is found to account for much of CP3's improvement on defensive APM this year, it would not fundamentally call into question the validity of the APM approach to player evaluation. Rather, it would highlight the need to utilize the APM model to test for player-by-player interaction effects to facilitate interpretation of the "main effects" that have been reported thus far.
Back to top

Ilardi



Joined: 15 May 2008
Posts: 265
Location: Lawrence, KS
Posted: Wed Feb 18, 2009 1:02 pm Post subject:

gabefarkas wrote:
Ilardi wrote:
gabefarkas wrote:
In that case, it seems the issue with Chris Paul is even more exaggerated, as he went from one extreme to the other, no?
No question: he's an outlier. It'll be interesting to see to what extent his high rating this season holds up through 82 games & playoffs.
I see. So you suspect it's likely that it may drop back down (towards 0) as more data is accumulated?
I'd bet on some regression to the mean, yes.
Back to top

Kevin Pelton
Site Admin


Joined: 30 Dec 2004
Posts: 979
Location: Seattle
Posted: Wed Feb 18, 2009 2:29 pm Post subject:

Ilardi wrote:
Paul's defensive APM estimate last season was -4.54 with a standard error of 1.52, so we can say with 95% confidence that his true rating last year was no better than -1.5 (since standard errors are roughly normally distributed). Conversely, his defensive APM this year (through Feb 3) is 6.76 with an se of 1.57, so we can say with 95% confidence that his true rating this year is no lower than roughly +3.5. So, despite the noise, his defensive APM rating this year is markedly improved, at a high level of statistical significance.
So the error terms you reported in the original post in this thread are strictly for defense? Do standard errors for offense/defense tend to be smaller than the overall errors?
Back to top

Mountain



Joined: 13 Mar 2007
Posts: 1527
Posted: Wed Feb 18, 2009 2:58 pm Post subject:

Lineup permutation and player interaction effects would be valuable new research tracks. Look forward to what you do / find / share Steve.
Back to top

Ilardi



Joined: 15 May 2008
Posts: 265
Location: Lawrence, KS
Posted: Wed Feb 18, 2009 3:03 pm Post subject:

Kevin Pelton wrote:
Ilardi wrote:
Paul's defensive APM estimate last season was -4.54 with a standard error of 1.52, so we can say with 95% confidence that his true rating last year was no better than -1.5 (since standard errors are roughly normally distributed). Conversely, his defensive APM this year (through Feb 3) is 6.76 with an se of 1.57, so we can say with 95% confidence that his true rating this year is no lower than roughly +3.5. So, despite the noise, his defensive APM rating this year is markedly improved, at a high level of statistical significance.
So the error terms you reported in the original post in this thread are strictly for defense? Do standard errors for offense/defense tend to be smaller than the overall errors?
They were standard errors that applied both to offensive and defensive APM estimates. (The two component errors always turn out to be virtually identical for any given player, with a correlation above r=.99, presumably because a player's lineup companions on offense and defense are almost exactly the same). The standard errors for overall (net offense + defense) APM are a little trickier, as there are two completely different ways they can be obtained (one with low error and one that's higher). If we start with a model that derives offensive and defensive ratings directly (with their associated errors), then we can calculate the overall APM error using the variance sum of squares law; it will be moderately (~40%) higher than the individual component (offense/defense) errors. Alternatively, we can go back and run an APM model to predict the overall APM ratings directly (rather than predicting the offensive/defensive components), in which case the overall APM ratings will have a smaller standard error, roughly on par with that of the individual offensive/defensive components (from the other model in which they were derived directly). Does that make sense? If so, what you'll find is that the error term I reported earlier in the thread can (in principle) be used as a close approximation of the standard error for the offensive, defensive, and overall APM estimates, respectively (keeping in mind that the error would apply to overall APM rating when estimated directly, rather than derived as the sum of offensive + defensive APM - fortunately, in practice, these two different estimates of overall APM tend to be very close to each other.)Last edited by Ilardi on Wed Feb 18, 2009 3:13 pm; edited 1 time in total
Back to top

Ilardi



Joined: 15 May 2008
Posts: 265
Location: Lawrence, KS
Posted: Wed Feb 18, 2009 3:12 pm Post subject:

Mountain wrote:
Lineup permutation and player interaction effects would be valuable new research tracks. Look forward to what you do / find / share Steve.
Thanks, me too!
Back to top

Ilardi



Joined: 15 May 2008
Posts: 265
Location: Lawrence, KS
Posted: Wed Feb 18, 2009 3:17 pm Post subject:

schtevie wrote:
Steve, there is one particular story of player interaction, raised early on in this string, that seems to me the most compelling to investigate. And this has to do with the interaction of Chris Paul playing alongside other small guards. In 2007-08, in almost 40% of his minutes, he was running with Pargo and Jackson. (And these same guys were his teammates in '06-'07.) Chris Paul is rated by one year Def. Adj+/- as being a liability compared to these two. Perhaps it was the case that Paul, being the notionally better defender, was guarding the 2, and taking one for the team. 82games counterpart data says he didn't do even a second of that, but that can't be right. This year, according to your data, Paul has played but 9 seconds with a guard smaller than 6' 4". This might be an important part of the turnaround. Would a precise way to address this issue be to divide CP into two players: Small Paul and Large Paul, where the former would be his minutes running with small 2s and the latter when he was not at the same risk of being abused by taller opponents?
P.S. Those aren't "my data". I think you're referring to Roland Beech's data on 82games. (I've published a few times on Roland's site, but that's my only affiliation.)
Back to top

bastillon



Joined: 04 Nov 2008
Posts: 55
Posted: Wed Feb 18, 2009 3:41 pm Post subject:

Quote:
(The two component errors always turn out to be virtually identical for any given player, with a correlation above r=.99, presumably because a player's lineup companions on offense and defense are almost exactly the same).
what about one-sided players like Nash(great offensively, liability defensively) or Ben Wallace(the opposite) ? if it's about companions, D and O should be completely separated because players' skills on both sides of the court can be completely different.


Page 6

Author Message Mountain



Joined: 13 Mar 2007
Posts: 1527
Posted: Wed Feb 18, 2009 3:43 pm Post subject:

If pure +/- is the main solution I think it will be pure +/- with an array of many splits and parts. As I've noted before Dan Rosenbaum introduced pure +/- and then seemed to immediately move to "overall +/-" with his statistical as about an 80% and pure at just 20%. I would really appreciate hearing from any the next pure adjusted stat producers if they considered following with a blended product or why they choose not to. And at this time I am no longer a fan of a 80% statistical 20% pure split because I tend to think this ends up shorting defense again. I wonder if something like 60% statistical / 40% pure or 50-70% statistical on offense depending on usage and therefore 30-50% pure offensive adjusted and maybe 40-50% statistical on defense and 50-60% pure defensive adjusted would be "better". A fuller blend feels more promising to me.
Back to top

Ilardi



Joined: 15 May 2008
Posts: 265
Location: Lawrence, KS
Posted: Wed Feb 18, 2009 3:48 pm Post subject:

bastillon wrote:
Quote:
(The two component errors always turn out to be virtually identical for any given player, with a correlation above r=.99, presumably because a player's lineup companions on offense and defense are almost exactly the same).
what about one-sided players like Nash(great offensively, liability defensively) or Ben Wallace(the opposite) ? if it's about companions, D and O should be completely separated because players' skills on both sides of the court can be completely different.
It's all about the lineups - estimation error generally results from heavily intercorrelated player minutes. Except for relatively infrequent, not-heavily-weighted one-possession lineups (where a lineup is used only for a single offensive or defensive possession), lineups tend to be identical for offensive and defensive possessions, so the multicollinearity issue for each player is roughly the same when estimating his offensive and defensive APM components.
Back to top

Ilardi



Joined: 15 May 2008
Posts: 265
Location: Lawrence, KS
Posted: Wed Feb 18, 2009 3:56 pm Post subject:

Mountain wrote:
If pure +/- is the main solution I think it will be pure +/- with an array of many splits and parts. As I've noted before Dan Rosenbaum introduced pure +/- and then seemed to immediately move to "overall +/-" with his statistical as about an 80% and pure at just 20%. I would really appreciate hearing from any the next pure adjusted stat producers if they considered following with a blended product or why they choose not to. And at this time I am no longer a fan of a 80% statistical 20% pure split because I tend to think this ends up shorting defense again. I wonder if something like 60% statistical / 40% pure or 50-70% statistical on offense depending on usage and therefore 30-50% pure offensive adjusted and maybe 40-50% statistical on defense and 50-60% pure defensive adjusted would be "better". A fuller blend feels more promising to me.
Dan's original "statistical +/-" relies entirely on traditional boxscore stats, and I seem to recall that it correlates pretty strongly with offensive APM (just as we'd expect) and not at all with defensive APM. I won't purport to speak for the others who've published APM analyses at some point (Aaron Barzilai, Dave Lewin, Eli Witus), but I have chosen to focus exclusively on "pure" APM for now because I believe the "statistical +/-" (SPM?) approach, as currently conceived, introduces unnecessary bias into the estimation process. However, as we've discussed in the past, I wouldn't rule out refining and augmenting the SPM approach in the future to make it more useful. (Knowing how innovative Dan is, he probably did all this back during the Clinton administration and just hasn't been able to make it publicly available.) Ultimately, however, my money is still on the pure APM measure (using multi-season databases to get estimation noise down to a reasonably low level), augmented with: (a) analysis of significant player-by-player interaction effects (probably rare, but potentially invaluable in the case of outlier APM results like CP3's), and (b) year-by-year time-trend analyses that compute individualized APM growth curve trajectories for each player.Last edited by Ilardi on Wed Feb 18, 2009 8:03 pm; edited 1 time in total
Back to top

gabefarkas



Joined: 31 Dec 2004
Posts: 1313
Location: Durham, NC
Posted: Wed Feb 18, 2009 5:35 pm Post subject:

Ilardi wrote:
2) The team's improvement in defensive efficiency with Paul on the court this season has also occurred despite a dramatic downturn in the team's defensive performance with Paul off the court. Last year, the team was better defensively when Paul was off-court (101.4) than when he was on (107.6); this year, the situation is reversed: they're much worse defensively when he's off-court (111.8 ) than when he's on (106.1). That's a swing of over 10 point per 100 possessions in Paul's teammates' off-court performance. And remember, most of these off-court teammates also share a high percentage of their minutes with Paul on-court. (We'll come back to this point in a moment).
It's circular logic. Either the team's OffEff and DefEff have changed because CP is a much better defender (as evidenced by his APM), or Paul's APM is much better because his team's OffEff and DefEff have changed. It's one or the other; you can't have both.
Back to top

gabefarkas



Joined: 31 Dec 2004
Posts: 1313
Location: Durham, NC
Posted: Wed Feb 18, 2009 5:38 pm Post subject:

schtevie wrote:
Perhaps it was the case that Paul, being the notionally better defender, was guarding the 2, and taking one for the team. 82games counterpart data says he didn't do even a second of that, but that can't be right.
No, it definitely can't be right! Especially if it doesn't support our pre-supposed conclusions!
Back to top

Ilardi



Joined: 15 May 2008
Posts: 265
Location: Lawrence, KS
Posted: Wed Feb 18, 2009 5:51 pm Post subject:

gabefarkas wrote:
Ilardi wrote:
2) The team's improvement in defensive efficiency with Paul on the court this season has also occurred despite a dramatic downturn in the team's defensive performance with Paul off the court. Last year, the team was better defensively when Paul was off-court (101.4) than when he was on (107.6); this year, the situation is reversed: they're much worse defensively when he's off-court (111.8 ) than when he's on (106.1). That's a swing of over 10 point per 100 possessions in Paul's teammates' off-court performance. And remember, most of these off-court teammates also share a high percentage of their minutes with Paul on-court. (We'll come back to this point in a moment).
It's circular logic. Either the team's OffEff and DefEff have changed because CP is a much better defender (as evidenced by his APM), or Paul's APM is much better because his team's OffEff and DefEff have changed. It's one or the other; you can't have both.
I don't follow you at all. What does the team's OffEff have to do with Paul's defensive APM? (That's what I was referring to, after all. His offensive APM is almost completely unchanged from 07-08 to 08-09.) And please recall that my contention in the passage you've quoted - admittedly somewhat subtle - is not about overall team DefEff, but with DefEff as it pertains to Paul's on-court and off-court minutes across the 07-08 and 08-09 seasons. If you can restate your objections, I'll be happy to try to respond . . .
Back to top

Ilardi



Joined: 15 May 2008
Posts: 265
Location: Lawrence, KS
Posted: Wed Feb 18, 2009 5:53 pm Post subject:

gabefarkas wrote:
schtevie wrote:
Perhaps it was the case that Paul, being the notionally better defender, was guarding the 2, and taking one for the team. 82games counterpart data says he didn't do even a second of that, but that can't be right.
No, it definitely can't be right! Especially if it doesn't support our pre-supposed conclusions!
Are you saying you believe Paul, who spent 40% of his 07-08 minutes playing in tandem with Pargo or Jackson, was never assigned to guard the 2-position?
Back to top

schtevie



Joined: 18 Apr 2005
Posts: 409
Posted: Wed Feb 18, 2009 6:43 pm Post subject:

I think Gabe's in a mood. What must be the case is that the counterpart positions on 82games are defined by offensive designation. No doubt that CP was the offensive point guard 100% of the time.
Back to top

Mountain



Joined: 13 Mar 2007
Posts: 1527
Posted: Thu Feb 19, 2009 3:06 am Post subject:

Alright Steve. Thanks again for the thoughtful replies.
Back to top

gabefarkas



Joined: 31 Dec 2004
Posts: 1313
Location: Durham, NC
Posted: Thu Feb 19, 2009 6:55 am Post subject:

Ilardi wrote:
gabefarkas wrote:
schtevie wrote:
Perhaps it was the case that Paul, being the notionally better defender, was guarding the 2, and taking one for the team. 82games counterpart data says he didn't do even a second of that, but that can't be right.
No, it definitely can't be right! Especially if it doesn't support our pre-supposed conclusions!
Are you saying you believe Paul, who spent 40% of his 07-08 minutes playing in tandem with Pargo or Jackson, was never assigned to guard the 2-position?
No, I'm just saying that it seems Schtevie is doing the same thing you and he accused Bob of doing a few days ago -- splitting hairs and twisting around conclusions. If it were me, and I found that 82games doesn't have data that supports my conclusion, I would keep looking elsewhere for other data that did, instead of just automatically assuming it's true, and stating something like "it must be true because I say so" (essentially what he wrote). Look, I'll lay my cards out on the table. I think APM is fascinating, and I want to learn more about what goes into it. My major gripe is twofold: (1) It's probably not as much of a panacea as its proponents seem to think it is. (2) Calling it "plus/minus" when it's split out into two parts, offensive and defensive, seems, for lack of a better word, silly. The very notion of a "plus/minus" implies that it accounts for the net difference from both ends of the court. Once you separate it out into two components, it ceases to contain that defining property.
Back to top

schtevie



Joined: 18 Apr 2005
Posts: 409
Posted: Thu Feb 19, 2009 8:19 am Post subject:

Gabe, I did not recently accuse Bob of "splitting hairs and twisting around conclusions". In fact I didn't say anything on the matter. OK, I thought about it, but I didn't. The fact remains that there is an interesting contrast between general perceptions of CP's defensive abilities and what APM says. It is an anomaly worth discussing. One factor explaining the gap might be the one I described. The numbers will be crunched at Steve's convenience and the relevance of the resulting estimates may or may not be confirmed by someone who could say whether in fact CP did in fact spend his time guarding 2s. Regarding APM generally, who says it is a panacea? Is there a better single metric, in concept, that is its superior? No, and by definition there cannot be. Underline it. Can the metric be improved by including age profiles, position adjustments, etc. Absolutely, and that is the (hopefully near-term) future. Finally, I don't understand your objection to dividing APM into Offensive and Defensive components. I just don't.
Back to top

gabefarkas



Joined: 31 Dec 2004
Posts: 1313
Location: Durham, NC
Posted: Thu Feb 19, 2009 9:16 am Post subject:

I appreciate your candor. If it appears our positions are intractable, as you describe, and you just don't understand my objection, then let's put this matter to rest. For the record, I'm not an opponent of APM. I'm just in general a skeptic.
Back to top

bchaikin



Joined: 27 Jan 2005
Posts: 687
Location: cleveland, ohio
Posted: Thu Feb 19, 2009 9:20 am Post subject:

The fact remains that there is an interesting contrast between general perceptions of CP's defensive abilities and what APM says. It is an anomaly worth discussing. not sure where the anomaly exists. you have a person who watched hornets games (in 07-08 ) tell us this about chris paul: I really don't understand all of this statistical stuff,but as someone who has actually watched most of the Hornets games I'm somewhat confused by the repeated claim that Jannero Pargo is a good defender or a better defender than Paul. Pargo is probably one of the worst defenders in the NBA and as a player he's utterly useless when his shot isn't falling. In fact I'm relatively sure that a lot of this discussion is a result of conflating Pargo's defense with Paul's. Before Pargo got hot and began to actually contribute I looked at 82games.com and Paul's defensive PER was around 14.5 or something. His differential PER was in the top five in the NBA because he was producing at better than twice the rate of opposing PG's. At that point Pargo was basically his backup and the Hornets hadn't yet given up on Morris Peterson. After Pargo began to produce he took much of Peterson's minutes,Pargo and Paul would probably play around 15 to 20 minutes per night together. Paul always guarded the stronger player because Pargo is completely incapable of dealing with larger players. That is to say that Pargo nearly always guarded the opposing point guard. Immediately Paul's defensive PER began to skyrocket.If you watched the games you'd see that teams go right at Pargo at every opportunity and that he offers very little resistance. and a person who charted hornets games tell us this about chris paul: Here are some numbers; The guy he was guarding scored .87 points per offensive action, (on 1203 actions). The best number on his team was .80 held by Rasual Butler (313 actions). Pargo's numbers were 1.01 on 620 actions. The numbers I gave were for 07-08. His defensive numbers for 07-08 were above average for the PG position, his numbers for this year are also above average for the PG position. and the people that quite possibly saw him play the most (coaches/sportswriters) vote him onto the all-D 2nd team and 2nd overall in MVP voting... but another person who neither watches chris paul ("extensively") nor charts hornets games tell us this: virtually everybody has trouble believing that a guy with eye-popping boxscore stats and horrific defense is really as big a liability as adjusted plus-minus says he is (think: Al Jefferson, Carmelo Anthony, CP3, etc.)... that's why adjusted plus-minus is so potentially valuable - by unmasking the fact that players like Calderon and Carmelo and Al Jefferson and Richard Jefferson and Amare Stoudemire don't have nearly the positive impact that people think they do, because they give away on defense much of what they add on offense (if not more, in some cases). Although Chris Paul was one of the Top 3 offensive players in the league (+9.24), he was actually a liability on the defensive side of the ball (-4.54). As a result, his overall contribution (+4.69), while impressive, was not commensurate with that of a top MVP candidate. yet later tries to modify his comments: In the thread on CP3's defense last year... I still concluded that he was a liability on defense in 07-08... but not to the extent that this completely "negated" his off-the-charts offensive contributions. it would be a mistake to look at a player's defensive APM number from a given season and conclude: this is a valid rating of Player X's defensive ability. Rather, it's merely an indicator of his defensive contribution over the period in question even if this interaction effect is found to account for much of CP3's improvement on defensive APM this year, it would not fundamentally call into question the validity of the APM approach to player evaluation. if anything the anomaly is the methodology, which i believe we all want to work... but when claims like the above are made, or: http://blogs.wsj.com/numbersguy/mark-cu ... mbers-580/ that "...sebastien telfair is more valuable than dirk nowitzki...", i can see why some may be skeptical or why someone might say adjusted +/- is "...one of the most overhyped player rating systems..."...
Back to top

schtevie



Joined: 18 Apr 2005
Posts: 409
Posted: Thu Feb 19, 2009 11:01 am Post subject:

Bob. To my surprise, I find that to be a rather reasonable summary of the discussion. I look forward to seeing what additional numbers Steve comes up with regarding interaction effects. As for APM being "...one of the most overhyped player rating systems..." however, I don't follow. First, I don't see much hype. Second, if you share this view, then you are at least implicitly of the belief that, overall, APM has shortcomings relative to another rating system. Is this true, or does your dissatisfaction relate only to particular estimates, such as CP's defensive APM in '07-'08 that don't pass your laugh test?
Back to top

deepak



Joined: 26 Apr 2006
Posts: 665
Posted: Thu Feb 19, 2009 11:17 am Post subject:

gabefarkas wrote:
Look, I'll lay my cards out on the table. I think APM is fascinating, and I want to learn more about what goes into it. My major gripe is twofold: (1) It's probably not as much of a panacea as its proponents seem to think it is. (2) Calling it "plus/minus" when it's split out into two parts, offensive and defensive, seems, for lack of a better word, silly. The very notion of a "plus/minus" implies that it accounts for the net difference from both ends of the court. Once you separate it out into two components, it ceases to contain that defining property.
I don't see the problem you're describing in (2). Plus/minus refers, generally speaking, to how the score changes with the player on the floor. Adjusted plus/minus is describing how the score changes assuming everyone else on the floor is an average player. Logically, this is the same as defining it as how the score changes with the player on the floor compared to if he was replaced by an average player, and everyone else stayed the same. So, if player A is +10 with everyone else on the floor being considered average, then that means if he replaces an average player and you make no other changes on the floor, he'd make the team +10 better. So, "adjusted plus/minus" and "on/off, replacing with an average player and holding everyone else constant" are really two sides of the same coin.
Back to top



Author Message Neil Paine



Joined: 13 Oct 2005
Posts: 774
Location: Atlanta, GA
Posted: Thu Feb 19, 2009 11:36 am Post subject:

schtevie wrote:
I don't see much hype.
Really? Because I think it's been considered a revolutionary metric that represents the future of APBRmetrics since well before this SI piece touting it ran in October 2005. And you yourself just got done saying this: Quote:
Is there a better single metric, in concept, that is its superior? No, and by definition there cannot be. Underline it.
So, yeah, there's been a lot of hype. Probably ever since the rumors about WINVAL first started to swirl in the early part of the decade. I like what Bob said when he wrote that "we all want APM to work". I desperately want it to live up to the hype. I want it to be that panacea. And maybe with further refinement it will come closer to being that. But it's unfortunate that it has known "hiccups", and that one of them happened to involve the best point guard in the league. Because I'm afraid this Chris Paul thing has discredited APM in the eyes of some people.
Back to top

schtevie



Joined: 18 Apr 2005
Posts: 397
Posted: Thu Feb 19, 2009 12:08 pm Post subject:

Referencing another current string, it was my impression from having recently listened to Bill Simmons chat with John Hollinger, that the host was not familiar with the concept. Or if he was, he did not mention it by name or potential. We are getting close to five years since Prof. Rosenbaum wrote his "Measuring How NBA Players Help Their Teams Win" article in 82games (and WINVAL has been around for how long?) and the word apparently hasn't gotten out. Let me more precise, I was referring to effective hype, not trees falling in the forest. And as for wanting APM to be a panacea, I don't know. What I want is a highly useful tool for making comparative judgements about particular basketball issues. Right now, I feel that the current version of APM does that better than any other I can think of. I don't expect that it can address all outstanding issues, and I expect future refinements that will make the tool more useful.
Back to top

bchaikin



Joined: 27 Jan 2005
Posts: 676
Location: cleveland, ohio
Posted: Thu Feb 19, 2009 12:16 pm Post subject:

As for APM being "...one of the most overhyped player rating systems..." however, I don't follow. not my quote, that came straight out of the WSJ article... First, I don't see much hype. perhaps you missed these gems from previous threads: defensive adjusted plus-minus is probably the single most valuable stat we have right now, since it picks up on something no other stat is capturing. Bottom line: adjusted plus-minus is enormously useful in player evaluation... In fact, I'm convinced it's the best single tool available... as for trying to convince us that the emperor has no clothes: statistical +/- tends to wildly overestimate the true adj. +/- value of poor defenders like Chris Paul it's still the case that CP3 is a below-average defender (indicated by his horrific defensive adj +/-) and I don't believe that assessment will change much based on multiple seasons' worth of data. unfortunately after awhile you just have to roll your eyes when the claims are so contrary to the available evidence, because... "...I'm afraid this Chris Paul thing has discredited APM in the eyes of some people...." not my words but i agree... Second, if you share this view, then you are at least implicitly of the belief that, overall, APM has shortcomings... some of it's results, yes... relative to another rating system... no, just other available evidence... Is this true, or does your dissatisfaction relate only to particular estimates, such as CP's defensive APM in '07-'08 that don't pass your laugh test?... yes to the latter... raw statistical +/- doesn't attempt portend to tell us anything definitive about an individual player. it's simply another piece of statistical evidence - which we are all grateful for - that when used in conjunction with other statistical evidence can help us to draw conclusions about players... adjusted +/- can be used in the same way. however some of it's adherents have simply attempted to draw and present conclusions about individual players as definitive without recognizing (or even acknowledging) other available evidence that is contrary to their findings...
Back to top

HoopStudies



Joined: 30 Dec 2004
Posts: 705
Location: Near Philadelphia, PA
Posted: Thu Feb 19, 2009 12:39 pm Post subject:

bchaikin wrote:
raw statistical +/- doesn't attempt portend to tell us anything definitive about an individual player. it's simply another piece of statistical evidence - which we are all grateful for - that when used in conjunction with other statistical evidence can help us to draw conclusions about players... adjusted +/- can be used in the same way. however some of it's adherents have simply attempted to draw and present conclusions about individual players as definitive without recognizing (or even acknowledging) other available evidence that is contrary to their findings...
I should note that the latter paragraph could be stated about adherents of almost any player rating method. Replace Adj +/- with tendex, points created, wins produced, etc. and you get another valid statement. When it comes to player rating methods, it seems that they evolve into religions rather than analysis tools. Bob is right that a lot of evidence gets thrown away or slowly (or immediately) ignored when discussing these things. It's a shame because a player rating should not be a crutch, something that you feel tells the truth. A player rating is compressing a lot of facts about a player (and his context) into a single solitary boring number. Given that the techniques are North-South-East-West of each other, there is no way that one number - any of them - can adequately tell the bigger more useful truth. The best use of these is to spawn the kind of debates that entertainment TV loves. I get the impression that so many shows have people arguing with people just because agreement is boring. At least player ratings give legitimate reasons for people to argue. The arguments may never accomplish a single thing, just like on TV, but at least people don't disagree because they feel that's more entertaining. I think that's why I've been reading this..._________________Dean Oliver Author, Basketball on Paper The postings are my own & don't necess represent positions, strategies or opinions of employers.
Back to top

Roland_Beech



Joined: 14 Jan 2005
Posts: 43
Posted: Thu Feb 19, 2009 12:39 pm Post subject:

So I have the ironic position of being someone who has published all kinds of articles by adjusted +/- advocates (Dan, Dave, Aaron, Steve), and yet I think the approach is fundamentally flawed and while not worthless, certainly something to be very skeptical of. My thing is that I don't believe players have 'constant +/- value' which in the end is what the regressions are trying to do. I am in Dean's camp which is that "fit" is all important and that the whole notion of constant value is ridiculous. Players do have capabilities, tendencies, traits, etc but a player's contribution to team success is governed by certainly the specific teammates he plays with (and opponents he is asked to match up with), the coaches, the coaching schemes, the role he is asked to fill, the current physical condition/injury status, the current mental/ emotional state, contract status etc A somewhat preposterous way to look at constant value is to imagine a 'cloned' team...if you cloned Ben Wallace and stuck five of him on the floor at the same time I would argue you would have the worst NBA team in history...yet if you surround him with four well fitted teammates, with a coach (Larry Brown) whose schemes and motivational tactics worked, at a time when Ben was healthy and perhaps feeling a little under-appreciated ...you can win a championship with him. So where is the consistent impact? (Dave Lewin's article had Ben as the 3rd best player in the league...Rip Hamilton as the WORST in 05-06 http://www.82games.com/lewin2.htm something he points out is a problem...) A recent point is the current Phoenix Suns...you look at the team and there's a ton of talent and yet they were ready to blow the whole roster apart a few days ago, only for Kerr to wise up it seems and decide that changing coach could produce immense change (I also disagree with Berri that "coaching doesn't matter") and I think it likely will, even before seeing the recent two explosive offensive games. Yet an adjusted +/- regression that treats the players as having uniform "state" across this season misses all kinds of nuance and detail before we even begin. Is Steve Nash post coaching change the same +/- value as Nash pre coaching change? Or how about Durant w/Carlesimo vs Brooks where he's been in different roles? Another example: Vince Carter has been a tremendous on/off player +5 in NJ '08-09 +12 in NJ '07-08 +9 in NJ '06-07 +15 in NJ '05-06 +11 in NJ '04-05 .... but -8 in TOR 04-05 http://www.82games.com/0405TOR.HTM +9 in TOR '03-04 Now it happened that Vince was unhappy and very vocal about wanting to be traded from Toronto in 04-05 right from the get go of the season...do ya think his mental state might have altered his usual 'constant' value some? Adjusted +/- completely misses a guy's emotional basis that changes, and completely misses physical issues...dealing with nagging injuries etc These points all serve to "invalidate" for me adjusted/regression +/- before we even get to bigger wholesale issues that arise and the "laugh test failures"...Dean's frescoball magical paper I think really frames the issue well for basketball and he was only looking at a two trait player...imagine what you have with basketball in terms of fit with loads of different skills you might classify and different roles for different positions, and different coaches asking different things of a player! The bigger question to me really is why people in the end think approaches that try to infer player contributions are better than simply tracking salient data to the point things are fairly self evident? It's safe to say that very soon data will be publicly available that will make much more transparent player individual defensive actions, shot creation, floor spacing, screen usage and screen setting, etc etc Why is there so much interest in turning to a mathematical construct for attempting to derive "statistical worth" when the raw material can be at hand to rather conclusively answer questions like "how good a help defender is Chris Paul?" or "how often does Paul attempting a steal hurt his team (vs the value of succeeding in the steal)?" or how about "on possessions where Devin Harris goes to the rim (and winds up in a heap under the basket), what is the change to his team's defensive pts/poss allowed coming back the other way?" To my mind there are going to be far more interesting stats to look at than regression outputs in the coming years... [I should also mention that while 82games has the unfortunately named 'roland rating' I wouldn't put much stock in that either...]
Back to top

Ryan J. Parker



Joined: 23 Mar 2007
Posts: 706
Location: Raleigh, NC
Posted: Thu Feb 19, 2009 1:08 pm Post subject:

Quote:
It's safe to say that very soon data will be publicly available that will make much more transparent player individual defensive actions, shot creation, floor spacing, screen usage and screen setting, etc
I figure the only guys that have this is Synergy, and they aren't (yet at least) just giving it away... I tried ripping some of it to no avail. I hope they have a change of heart. _________________I am a basketball geek.
Back to top

Neil Paine



Joined: 13 Oct 2005
Posts: 774
Location: Atlanta, GA
Posted: Thu Feb 19, 2009 1:25 pm Post subject:

Great posts, Roland & Dean. I could not agree more. There is no such thing as constant value -- as you mention, the same player can have radically different values to different teams, in different systems. "Fit" is hugely important, and we all but ignore that when we distill everything into a single, supposedly "context-free" number. I don't think any evaluatory method is going to be that panacea until there is some mechanism by which you can effectively predict the performance of any 5-man unit ahead of time (including a 5-man unit of all low-usage or all high-usage players).
Back to top

Mountain



Joined: 13 Mar 2007
Posts: 1527
Posted: Thu Feb 19, 2009 2:48 pm Post subject:

davis21wylie2121 wrote:
I don't think any evaluatory method is going to be that panacea until there is some mechanism by which you can effectively predict the performance of any 5-man unit ahead of time .
And my bet it is going to the adjusted approach fully applied to lineups that will get us closest. Players aren't constant in a range of 100+ lineups but lineups are lineups. Now opponents and style of play and schedule consideration vary but the 5 man lineup is as much stability as you'll ever get in basketball. Right now raw lineup data is what all but a few or one team sees on a current basis. Even that can be fairly helpful. I think taking a big minute success lineup and making it even bigger is the simplest available predictor of future lineup performance, short of having lineup adjusted, hence my advocacy for riding the winners you know pretty well even harder than usual like the Celtics do or the championship Pistons did, over all these spur of the moment scattered minute small lineups that all teams seem to have too much of in my opinion. Use the regular season to understand a few dozen lineups as well as you can. Pure adjusted in simplest form is not perfect and that is often noted by the authors, but it is important to acknowledge that we are generally much better off with it than without it. It recognizes most of the very best. It identifies many of the suspected bad wheels in a way they we didn't have before. It probably sorts most in the middle fairly well. 60-70% of the ratings are probably pretty close to what it estimates. Yes the outliers vex and can serve to push the method forward. We should give it time to move into its second phase and various forms including most importantly lineup adjusted. Multi-year and offense/defense has its place but it is probably not the end of the line. I think we will gain with publicly available statistical/pure blends, adjusted against playoff level teams and other style characterizations, adjusted in just the playoffs, adjusted player pair or triplets (representing cores that most lineups revolve around), adjusted lineups and multi-level cuts like adjusted lineup / against playoff level opponents / offense-defense split and maybe even that take to multi-season (at least this and last season and the previous playoff or two).Last edited by Mountain on Thu Feb 19, 2009 3:24 pm; edited 2 times in total
Back to top

schtevie



Joined: 18 Apr 2005
Posts: 397
Posted: Thu Feb 19, 2009 2:53 pm Post subject:

Dean, My strong sense is that you are being too agnostic by half. A player rating may be a single number purporting to represent "overall" value (and actual context, of course) but it is not necessarily boring, in the sense that there are important issues to which a single number can provide interesting and relevant insight (assuming that it is accurate, of course). A case in point to which I have recently referred is the value of Tony Allen. There are a lot of clever people out there who seem to feel that he is a bum. I cannot recall reading any published opinion (though I haven't searched for any, to be honest) that considers him to be a really good player. And his relatively low salary would seem to bear out this conventional wisdom. There is, however, a metric which is praiseworthy in its construction, though not without some flaws, that identifies him as a really good player. Particularly on the defensive end. Assuming this rating - this notionally boring number - is accurate, how would knowledge of this fact, by one or by all, not "accomplish a single thing". More generally, what is the bigger, more useful, truth that you feel is beyond reach? If your point is that there are more interesting questions to ask and deeper truths available that can only be answered by peeling back the layers of the onion, well, OK. But that is a separate issue.
Back to top

Mountain



Joined: 13 Mar 2007
Posts: 1527
Posted: Thu Feb 19, 2009 3:20 pm Post subject:

T Allen is a player pure +/- has never given a negative to. Generally nice positive, one neutral , one wildly high. I believe in Boston, in the role he has been given that he, on average, doesn't hurt and actually helps some and gets too much flak. But there are probably things he tries and shouldn't from time to time and good and weak lineups. So the pure adjusted is legitimizing on the roster but not the end of the management. There are teams I would have urged pursue T Allen now or later and heavily supported by the value measurement from pure adjusted.
Back to top

Kevin Pelton
Site Admin


Joined: 30 Dec 2004
Posts: 975
Location: Seattle
Posted: Thu Feb 19, 2009 3:57 pm Post subject:

schtevie wrote:
Assuming this rating - this notionally boring number - is accurate, how would knowledge of this fact, by one or by all, not "accomplish a single thing".
But that is precisely the issue! What evidence exists to justify the assumption that adjusted-plus minus is right and everything else is wrong? Are we to go strictly by the success of last year's Celtics?
Back to top

basketballvalue



Joined: 07 Mar 2006
Posts: 204
Posted: Thu Feb 19, 2009 4:11 pm Post subject:

HoopStudies wrote:
A player rating is compressing a lot of facts about a player (and his context) into a single solitary boring number. Given that the techniques are North-South-East-West of each other, there is no way that one number - any of them - can adequately tell the bigger more useful truth.
Roland_Beech wrote:
So I have the ironic position of being someone who has published all kinds of articles by adjusted +/- advocates (Dan, Dave, Aaron, Steve), and yet I think the approach is fundamentally flawed and while not worthless, certainly something to be very skeptical of. My thing is that I don't believe players have 'constant +/- value' which in the end is what the regressions are trying to do. I am in Dean's camp which is that "fit" is all important and that the whole notion of constant value is ridiculous.
Good posts everybody, I'm enjoying the thread even though people seem to be getting pretty fired up. We're all friends, right? I completely agree with your point, Dean, and have tried to emphasize that in the use of +/-. As far as your post, Roland, I also agree that people don't have a constant +/- value that just moves from team to team when player's change teams. Would you say the same thing about most other stats out there except perhaps FT% and a few others? Certainly I would think the constant value comment applies to PER and many more (including Roland Rating :]). If not, I'd really be intrigued to hear more of your thoughts about that. Thanks, Aaron PS I think it's pretty clear, but I hope people haven't confused the fact that just because I'm calculating and publishing +/- that I'm it's biggest proponent and think we should ignore all other information. It's just where I felt I could make my biggest contribution to the field when I started. I always consider other metrics when evaluating individual players._________________www.basketballvalue.com Follow on Twitter
Back to top

HoopStudies



Joined: 30 Dec 2004
Posts: 705
Location: Near Philadelphia, PA
Posted: Thu Feb 19, 2009 6:40 pm Post subject:

schtevie wrote:
Dean, My strong sense is that you are being too agnostic by half. A player rating may be a single number purporting to represent "overall" value (and actual context, of course) but it is not necessarily boring, in the sense that there are important issues to which a single number can provide interesting and relevant insight (assuming that it is accurate, of course).
So you're saying that I was right except for the "boring" part. I can live with that. I do admittedly love numbers, so maybe I hurt their feelings by calling them boring. I'm not saying that a summary rating has no value, just far less than we place on it by believing them so strongly (speaking of all metrics). schtevie wrote:
More generally, what is the bigger, more useful, truth that you feel is beyond reach?
Answering this the right way is one step towards getting a job doing it._________________Dean Oliver Author, Basketball on Paper The postings are my own & don't necess represent positions, strategies or opinions of employers.
Back to top

mtamada



Joined: 28 Jan 2005
Posts: 375
Posted: Thu Feb 19, 2009 6:54 pm Post subject:

davis21wylie2121 wrote:
I don't think any evaluatory method is going to be that panacea until there is some mechanism by which you can effectively predict the performance of any 5-man unit ahead of time (including a 5-man unit of all low-usage or all high-usage players).
That sets the bar perhaps a tad too high, but yup all along that is what I have said is the Holy Grail of basketball stats. The reason why I say it might be too demanding is that in practical terms (both for making useful predictions, and for having worthwhile data to base estimates upon), we will not often see those extreme hypothetical lineups that you describe, of 5 high-usage players, or 5 cloned Ben Wallaces, to use another example that has been mentioned. But aside from that, right on. Even avoiding those extreme situations, there are difficulties including: agreeing upon what an "effective" prediction is. E.g. is it even possible to take into account not just a player's role or fit with teammates, but the coaching style and decisions. Dick Motta seemingly underused both Dale Ellis and Detlef Schrempf with the Mavs. Bill Simmons has pointed out that D'Antoni's SSOL strategy, whatever else its merits, may be inflating the stats of the players who played in it: Nash, Marion, likely Stoudemire, etc. I.e. it may be an impossible task to predict the performance of a 5-man unit, if coaching strategies and decisions have significant impacts on what those 5 players do. Oh, one game where we do see 5 high-usage players on the floor at once: the All-Star game. But that game is so different from others, that I would not expect even the best stats to have much predictive value. (Might be an interesting test: maybe I'm wrong. Maybe there is a wonderful consistency to players' abilities as measured by whatever we think the best measure is. Then All-star game stats could be used to prove it -- a natural experiment! My strong hunch though is that all-star games are such a different environment that performance in those games will be difficult to predict effecitvely.) I guess the Dream Team, and Redeem Team, and other Olympic squads would be other examples of 5 high-usage players on the court simultaneously. The US teams' penchants for either destroying the opposition, or falling flat on their face (6th place in the 2002 World Championships and eking out a bronze in 2004) however suggests again that predictability is probably low.
Back to top

mtamada



Joined: 28 Jan 2005
Posts: 375
Posted: Thu Feb 19, 2009 7:16 pm Post subject:

mtamada wrote:
suggests again that predictability is probably low.
I should clarify that I think predictability is low for All-star games and probably Olympic games. For NBA teams, predictability is probably higher ... especially if the same coach (and generally similar coaching strategies) are employed from this season to the next.
Back to top


Top
 Profile  
 
PostPosted: Fri Apr 15, 2011 5:50 pm 
Offline

Joined: Thu Apr 14, 2011 11:10 pm
Posts: 2548
Page 6

Author Message Mountain



Joined: 13 Mar 2007
Posts: 1527
Posted: Wed Feb 18, 2009 3:43 pm Post subject:

If pure +/- is the main solution I think it will be pure +/- with an array of many splits and parts. As I've noted before Dan Rosenbaum introduced pure +/- and then seemed to immediately move to "overall +/-" with his statistical as about an 80% and pure at just 20%. I would really appreciate hearing from any the next pure adjusted stat producers if they considered following with a blended product or why they choose not to. And at this time I am no longer a fan of a 80% statistical 20% pure split because I tend to think this ends up shorting defense again. I wonder if something like 60% statistical / 40% pure or 50-70% statistical on offense depending on usage and therefore 30-50% pure offensive adjusted and maybe 40-50% statistical on defense and 50-60% pure defensive adjusted would be "better". A fuller blend feels more promising to me.
Back to top

Ilardi



Joined: 15 May 2008
Posts: 265
Location: Lawrence, KS
Posted: Wed Feb 18, 2009 3:48 pm Post subject:

bastillon wrote:
Quote:
(The two component errors always turn out to be virtually identical for any given player, with a correlation above r=.99, presumably because a player's lineup companions on offense and defense are almost exactly the same).
what about one-sided players like Nash(great offensively, liability defensively) or Ben Wallace(the opposite) ? if it's about companions, D and O should be completely separated because players' skills on both sides of the court can be completely different.
It's all about the lineups - estimation error generally results from heavily intercorrelated player minutes. Except for relatively infrequent, not-heavily-weighted one-possession lineups (where a lineup is used only for a single offensive or defensive possession), lineups tend to be identical for offensive and defensive possessions, so the multicollinearity issue for each player is roughly the same when estimating his offensive and defensive APM components.
Back to top

Ilardi



Joined: 15 May 2008
Posts: 265
Location: Lawrence, KS
Posted: Wed Feb 18, 2009 3:56 pm Post subject:

Mountain wrote:
If pure +/- is the main solution I think it will be pure +/- with an array of many splits and parts. As I've noted before Dan Rosenbaum introduced pure +/- and then seemed to immediately move to "overall +/-" with his statistical as about an 80% and pure at just 20%. I would really appreciate hearing from any the next pure adjusted stat producers if they considered following with a blended product or why they choose not to. And at this time I am no longer a fan of a 80% statistical 20% pure split because I tend to think this ends up shorting defense again. I wonder if something like 60% statistical / 40% pure or 50-70% statistical on offense depending on usage and therefore 30-50% pure offensive adjusted and maybe 40-50% statistical on defense and 50-60% pure defensive adjusted would be "better". A fuller blend feels more promising to me.
Dan's original "statistical +/-" relies entirely on traditional boxscore stats, and I seem to recall that it correlates pretty strongly with offensive APM (just as we'd expect) and not at all with defensive APM. I won't purport to speak for the others who've published APM analyses at some point (Aaron Barzilai, Dave Lewin, Eli Witus), but I have chosen to focus exclusively on "pure" APM for now because I believe the "statistical +/-" (SPM?) approach, as currently conceived, introduces unnecessary bias into the estimation process. However, as we've discussed in the past, I wouldn't rule out refining and augmenting the SPM approach in the future to make it more useful. (Knowing how innovative Dan is, he probably did all this back during the Clinton administration and just hasn't been able to make it publicly available.) Ultimately, however, my money is still on the pure APM measure (using multi-season databases to get estimation noise down to a reasonably low level), augmented with: (a) analysis of significant player-by-player interaction effects (probably rare, but potentially invaluable in the case of outlier APM results like CP3's), and (b) year-by-year time-trend analyses that compute individualized APM growth curve trajectories for each player.Last edited by Ilardi on Wed Feb 18, 2009 8:03 pm; edited 1 time in total
Back to top

gabefarkas



Joined: 31 Dec 2004
Posts: 1313
Location: Durham, NC
Posted: Wed Feb 18, 2009 5:35 pm Post subject:

Ilardi wrote:
2) The team's improvement in defensive efficiency with Paul on the court this season has also occurred despite a dramatic downturn in the team's defensive performance with Paul off the court. Last year, the team was better defensively when Paul was off-court (101.4) than when he was on (107.6); this year, the situation is reversed: they're much worse defensively when he's off-court (111.8 ) than when he's on (106.1). That's a swing of over 10 point per 100 possessions in Paul's teammates' off-court performance. And remember, most of these off-court teammates also share a high percentage of their minutes with Paul on-court. (We'll come back to this point in a moment).
It's circular logic. Either the team's OffEff and DefEff have changed because CP is a much better defender (as evidenced by his APM), or Paul's APM is much better because his team's OffEff and DefEff have changed. It's one or the other; you can't have both.
Back to top

gabefarkas



Joined: 31 Dec 2004
Posts: 1313
Location: Durham, NC
Posted: Wed Feb 18, 2009 5:38 pm Post subject:

schtevie wrote:
Perhaps it was the case that Paul, being the notionally better defender, was guarding the 2, and taking one for the team. 82games counterpart data says he didn't do even a second of that, but that can't be right.
No, it definitely can't be right! Especially if it doesn't support our pre-supposed conclusions!
Back to top

Ilardi



Joined: 15 May 2008
Posts: 265
Location: Lawrence, KS
Posted: Wed Feb 18, 2009 5:51 pm Post subject:

gabefarkas wrote:
Ilardi wrote:
2) The team's improvement in defensive efficiency with Paul on the court this season has also occurred despite a dramatic downturn in the team's defensive performance with Paul off the court. Last year, the team was better defensively when Paul was off-court (101.4) than when he was on (107.6); this year, the situation is reversed: they're much worse defensively when he's off-court (111.8 ) than when he's on (106.1). That's a swing of over 10 point per 100 possessions in Paul's teammates' off-court performance. And remember, most of these off-court teammates also share a high percentage of their minutes with Paul on-court. (We'll come back to this point in a moment).
It's circular logic. Either the team's OffEff and DefEff have changed because CP is a much better defender (as evidenced by his APM), or Paul's APM is much better because his team's OffEff and DefEff have changed. It's one or the other; you can't have both.
I don't follow you at all. What does the team's OffEff have to do with Paul's defensive APM? (That's what I was referring to, after all. His offensive APM is almost completely unchanged from 07-08 to 08-09.) And please recall that my contention in the passage you've quoted - admittedly somewhat subtle - is not about overall team DefEff, but with DefEff as it pertains to Paul's on-court and off-court minutes across the 07-08 and 08-09 seasons. If you can restate your objections, I'll be happy to try to respond . . .
Back to top

Ilardi



Joined: 15 May 2008
Posts: 265
Location: Lawrence, KS
Posted: Wed Feb 18, 2009 5:53 pm Post subject:

gabefarkas wrote:
schtevie wrote:
Perhaps it was the case that Paul, being the notionally better defender, was guarding the 2, and taking one for the team. 82games counterpart data says he didn't do even a second of that, but that can't be right.
No, it definitely can't be right! Especially if it doesn't support our pre-supposed conclusions!
Are you saying you believe Paul, who spent 40% of his 07-08 minutes playing in tandem with Pargo or Jackson, was never assigned to guard the 2-position?
Back to top

schtevie



Joined: 18 Apr 2005
Posts: 409
Posted: Wed Feb 18, 2009 6:43 pm Post subject:

I think Gabe's in a mood. What must be the case is that the counterpart positions on 82games are defined by offensive designation. No doubt that CP was the offensive point guard 100% of the time.
Back to top

Mountain



Joined: 13 Mar 2007
Posts: 1527
Posted: Thu Feb 19, 2009 3:06 am Post subject:

Alright Steve. Thanks again for the thoughtful replies.
Back to top

gabefarkas



Joined: 31 Dec 2004
Posts: 1313
Location: Durham, NC
Posted: Thu Feb 19, 2009 6:55 am Post subject:

Ilardi wrote:
gabefarkas wrote:
schtevie wrote:
Perhaps it was the case that Paul, being the notionally better defender, was guarding the 2, and taking one for the team. 82games counterpart data says he didn't do even a second of that, but that can't be right.
No, it definitely can't be right! Especially if it doesn't support our pre-supposed conclusions!
Are you saying you believe Paul, who spent 40% of his 07-08 minutes playing in tandem with Pargo or Jackson, was never assigned to guard the 2-position?
No, I'm just saying that it seems Schtevie is doing the same thing you and he accused Bob of doing a few days ago -- splitting hairs and twisting around conclusions. If it were me, and I found that 82games doesn't have data that supports my conclusion, I would keep looking elsewhere for other data that did, instead of just automatically assuming it's true, and stating something like "it must be true because I say so" (essentially what he wrote). Look, I'll lay my cards out on the table. I think APM is fascinating, and I want to learn more about what goes into it. My major gripe is twofold: (1) It's probably not as much of a panacea as its proponents seem to think it is. (2) Calling it "plus/minus" when it's split out into two parts, offensive and defensive, seems, for lack of a better word, silly. The very notion of a "plus/minus" implies that it accounts for the net difference from both ends of the court. Once you separate it out into two components, it ceases to contain that defining property.
Back to top

schtevie



Joined: 18 Apr 2005
Posts: 409
Posted: Thu Feb 19, 2009 8:19 am Post subject:

Gabe, I did not recently accuse Bob of "splitting hairs and twisting around conclusions". In fact I didn't say anything on the matter. OK, I thought about it, but I didn't. The fact remains that there is an interesting contrast between general perceptions of CP's defensive abilities and what APM says. It is an anomaly worth discussing. One factor explaining the gap might be the one I described. The numbers will be crunched at Steve's convenience and the relevance of the resulting estimates may or may not be confirmed by someone who could say whether in fact CP did in fact spend his time guarding 2s. Regarding APM generally, who says it is a panacea? Is there a better single metric, in concept, that is its superior? No, and by definition there cannot be. Underline it. Can the metric be improved by including age profiles, position adjustments, etc. Absolutely, and that is the (hopefully near-term) future. Finally, I don't understand your objection to dividing APM into Offensive and Defensive components. I just don't.
Back to top

gabefarkas



Joined: 31 Dec 2004
Posts: 1313
Location: Durham, NC
Posted: Thu Feb 19, 2009 9:16 am Post subject:

I appreciate your candor. If it appears our positions are intractable, as you describe, and you just don't understand my objection, then let's put this matter to rest. For the record, I'm not an opponent of APM. I'm just in general a skeptic.
Back to top

bchaikin



Joined: 27 Jan 2005
Posts: 687
Location: cleveland, ohio
Posted: Thu Feb 19, 2009 9:20 am Post subject:

The fact remains that there is an interesting contrast between general perceptions of CP's defensive abilities and what APM says. It is an anomaly worth discussing. not sure where the anomaly exists. you have a person who watched hornets games (in 07-08 ) tell us this about chris paul: I really don't understand all of this statistical stuff,but as someone who has actually watched most of the Hornets games I'm somewhat confused by the repeated claim that Jannero Pargo is a good defender or a better defender than Paul. Pargo is probably one of the worst defenders in the NBA and as a player he's utterly useless when his shot isn't falling. In fact I'm relatively sure that a lot of this discussion is a result of conflating Pargo's defense with Paul's. Before Pargo got hot and began to actually contribute I looked at 82games.com and Paul's defensive PER was around 14.5 or something. His differential PER was in the top five in the NBA because he was producing at better than twice the rate of opposing PG's. At that point Pargo was basically his backup and the Hornets hadn't yet given up on Morris Peterson. After Pargo began to produce he took much of Peterson's minutes,Pargo and Paul would probably play around 15 to 20 minutes per night together. Paul always guarded the stronger player because Pargo is completely incapable of dealing with larger players. That is to say that Pargo nearly always guarded the opposing point guard. Immediately Paul's defensive PER began to skyrocket.If you watched the games you'd see that teams go right at Pargo at every opportunity and that he offers very little resistance. and a person who charted hornets games tell us this about chris paul: Here are some numbers; The guy he was guarding scored .87 points per offensive action, (on 1203 actions). The best number on his team was .80 held by Rasual Butler (313 actions). Pargo's numbers were 1.01 on 620 actions. The numbers I gave were for 07-08. His defensive numbers for 07-08 were above average for the PG position, his numbers for this year are also above average for the PG position. and the people that quite possibly saw him play the most (coaches/sportswriters) vote him onto the all-D 2nd team and 2nd overall in MVP voting... but another person who neither watches chris paul ("extensively") nor charts hornets games tell us this: virtually everybody has trouble believing that a guy with eye-popping boxscore stats and horrific defense is really as big a liability as adjusted plus-minus says he is (think: Al Jefferson, Carmelo Anthony, CP3, etc.)... that's why adjusted plus-minus is so potentially valuable - by unmasking the fact that players like Calderon and Carmelo and Al Jefferson and Richard Jefferson and Amare Stoudemire don't have nearly the positive impact that people think they do, because they give away on defense much of what they add on offense (if not more, in some cases). Although Chris Paul was one of the Top 3 offensive players in the league (+9.24), he was actually a liability on the defensive side of the ball (-4.54). As a result, his overall contribution (+4.69), while impressive, was not commensurate with that of a top MVP candidate. yet later tries to modify his comments: In the thread on CP3's defense last year... I still concluded that he was a liability on defense in 07-08... but not to the extent that this completely "negated" his off-the-charts offensive contributions. it would be a mistake to look at a player's defensive APM number from a given season and conclude: this is a valid rating of Player X's defensive ability. Rather, it's merely an indicator of his defensive contribution over the period in question even if this interaction effect is found to account for much of CP3's improvement on defensive APM this year, it would not fundamentally call into question the validity of the APM approach to player evaluation. if anything the anomaly is the methodology, which i believe we all want to work... but when claims like the above are made, or: http://blogs.wsj.com/numbersguy/mark-cu ... mbers-580/ that "...sebastien telfair is more valuable than dirk nowitzki...", i can see why some may be skeptical or why someone might say adjusted +/- is "...one of the most overhyped player rating systems..."...
Back to top

schtevie



Joined: 18 Apr 2005
Posts: 409
Posted: Thu Feb 19, 2009 11:01 am Post subject:

Bob. To my surprise, I find that to be a rather reasonable summary of the discussion. I look forward to seeing what additional numbers Steve comes up with regarding interaction effects. As for APM being "...one of the most overhyped player rating systems..." however, I don't follow. First, I don't see much hype. Second, if you share this view, then you are at least implicitly of the belief that, overall, APM has shortcomings relative to another rating system. Is this true, or does your dissatisfaction relate only to particular estimates, such as CP's defensive APM in '07-'08 that don't pass your laugh test?
Back to top

deepak



Joined: 26 Apr 2006
Posts: 665
Posted: Thu Feb 19, 2009 11:17 am Post subject:

gabefarkas wrote:
Look, I'll lay my cards out on the table. I think APM is fascinating, and I want to learn more about what goes into it. My major gripe is twofold: (1) It's probably not as much of a panacea as its proponents seem to think it is. (2) Calling it "plus/minus" when it's split out into two parts, offensive and defensive, seems, for lack of a better word, silly. The very notion of a "plus/minus" implies that it accounts for the net difference from both ends of the court. Once you separate it out into two components, it ceases to contain that defining property.
I don't see the problem you're describing in (2). Plus/minus refers, generally speaking, to how the score changes with the player on the floor. Adjusted plus/minus is describing how the score changes assuming everyone else on the floor is an average player. Logically, this is the same as defining it as how the score changes with the player on the floor compared to if he was replaced by an average player, and everyone else stayed the same. So, if player A is +10 with everyone else on the floor being considered average, then that means if he replaces an average player and you make no other changes on the floor, he'd make the team +10 better. So, "adjusted plus/minus" and "on/off, replacing with an average player and holding everyone else constant" are really two sides of the same coin.
Back to top



Author Message Neil Paine



Joined: 13 Oct 2005
Posts: 774
Location: Atlanta, GA
Posted: Thu Feb 19, 2009 11:36 am Post subject:

schtevie wrote:
I don't see much hype.
Really? Because I think it's been considered a revolutionary metric that represents the future of APBRmetrics since well before this SI piece touting it ran in October 2005. And you yourself just got done saying this: Quote:
Is there a better single metric, in concept, that is its superior? No, and by definition there cannot be. Underline it.
So, yeah, there's been a lot of hype. Probably ever since the rumors about WINVAL first started to swirl in the early part of the decade. I like what Bob said when he wrote that "we all want APM to work". I desperately want it to live up to the hype. I want it to be that panacea. And maybe with further refinement it will come closer to being that. But it's unfortunate that it has known "hiccups", and that one of them happened to involve the best point guard in the league. Because I'm afraid this Chris Paul thing has discredited APM in the eyes of some people.
Back to top

schtevie



Joined: 18 Apr 2005
Posts: 397
Posted: Thu Feb 19, 2009 12:08 pm Post subject:

Referencing another current string, it was my impression from having recently listened to Bill Simmons chat with John Hollinger, that the host was not familiar with the concept. Or if he was, he did not mention it by name or potential. We are getting close to five years since Prof. Rosenbaum wrote his "Measuring How NBA Players Help Their Teams Win" article in 82games (and WINVAL has been around for how long?) and the word apparently hasn't gotten out. Let me more precise, I was referring to effective hype, not trees falling in the forest. And as for wanting APM to be a panacea, I don't know. What I want is a highly useful tool for making comparative judgements about particular basketball issues. Right now, I feel that the current version of APM does that better than any other I can think of. I don't expect that it can address all outstanding issues, and I expect future refinements that will make the tool more useful.
Back to top

bchaikin



Joined: 27 Jan 2005
Posts: 676
Location: cleveland, ohio
Posted: Thu Feb 19, 2009 12:16 pm Post subject:

As for APM being "...one of the most overhyped player rating systems..." however, I don't follow. not my quote, that came straight out of the WSJ article... First, I don't see much hype. perhaps you missed these gems from previous threads: defensive adjusted plus-minus is probably the single most valuable stat we have right now, since it picks up on something no other stat is capturing. Bottom line: adjusted plus-minus is enormously useful in player evaluation... In fact, I'm convinced it's the best single tool available... as for trying to convince us that the emperor has no clothes: statistical +/- tends to wildly overestimate the true adj. +/- value of poor defenders like Chris Paul it's still the case that CP3 is a below-average defender (indicated by his horrific defensive adj +/-) and I don't believe that assessment will change much based on multiple seasons' worth of data. unfortunately after awhile you just have to roll your eyes when the claims are so contrary to the available evidence, because... "...I'm afraid this Chris Paul thing has discredited APM in the eyes of some people...." not my words but i agree... Second, if you share this view, then you are at least implicitly of the belief that, overall, APM has shortcomings... some of it's results, yes... relative to another rating system... no, just other available evidence... Is this true, or does your dissatisfaction relate only to particular estimates, such as CP's defensive APM in '07-'08 that don't pass your laugh test?... yes to the latter... raw statistical +/- doesn't attempt portend to tell us anything definitive about an individual player. it's simply another piece of statistical evidence - which we are all grateful for - that when used in conjunction with other statistical evidence can help us to draw conclusions about players... adjusted +/- can be used in the same way. however some of it's adherents have simply attempted to draw and present conclusions about individual players as definitive without recognizing (or even acknowledging) other available evidence that is contrary to their findings...
Back to top

HoopStudies



Joined: 30 Dec 2004
Posts: 705
Location: Near Philadelphia, PA
Posted: Thu Feb 19, 2009 12:39 pm Post subject:

bchaikin wrote:
raw statistical +/- doesn't attempt portend to tell us anything definitive about an individual player. it's simply another piece of statistical evidence - which we are all grateful for - that when used in conjunction with other statistical evidence can help us to draw conclusions about players... adjusted +/- can be used in the same way. however some of it's adherents have simply attempted to draw and present conclusions about individual players as definitive without recognizing (or even acknowledging) other available evidence that is contrary to their findings...
I should note that the latter paragraph could be stated about adherents of almost any player rating method. Replace Adj +/- with tendex, points created, wins produced, etc. and you get another valid statement. When it comes to player rating methods, it seems that they evolve into religions rather than analysis tools. Bob is right that a lot of evidence gets thrown away or slowly (or immediately) ignored when discussing these things. It's a shame because a player rating should not be a crutch, something that you feel tells the truth. A player rating is compressing a lot of facts about a player (and his context) into a single solitary boring number. Given that the techniques are North-South-East-West of each other, there is no way that one number - any of them - can adequately tell the bigger more useful truth. The best use of these is to spawn the kind of debates that entertainment TV loves. I get the impression that so many shows have people arguing with people just because agreement is boring. At least player ratings give legitimate reasons for people to argue. The arguments may never accomplish a single thing, just like on TV, but at least people don't disagree because they feel that's more entertaining. I think that's why I've been reading this..._________________Dean Oliver Author, Basketball on Paper The postings are my own & don't necess represent positions, strategies or opinions of employers.
Back to top

Roland_Beech



Joined: 14 Jan 2005
Posts: 43
Posted: Thu Feb 19, 2009 12:39 pm Post subject:

So I have the ironic position of being someone who has published all kinds of articles by adjusted +/- advocates (Dan, Dave, Aaron, Steve), and yet I think the approach is fundamentally flawed and while not worthless, certainly something to be very skeptical of. My thing is that I don't believe players have 'constant +/- value' which in the end is what the regressions are trying to do. I am in Dean's camp which is that "fit" is all important and that the whole notion of constant value is ridiculous. Players do have capabilities, tendencies, traits, etc but a player's contribution to team success is governed by certainly the specific teammates he plays with (and opponents he is asked to match up with), the coaches, the coaching schemes, the role he is asked to fill, the current physical condition/injury status, the current mental/ emotional state, contract status etc A somewhat preposterous way to look at constant value is to imagine a 'cloned' team...if you cloned Ben Wallace and stuck five of him on the floor at the same time I would argue you would have the worst NBA team in history...yet if you surround him with four well fitted teammates, with a coach (Larry Brown) whose schemes and motivational tactics worked, at a time when Ben was healthy and perhaps feeling a little under-appreciated ...you can win a championship with him. So where is the consistent impact? (Dave Lewin's article had Ben as the 3rd best player in the league...Rip Hamilton as the WORST in 05-06 http://www.82games.com/lewin2.htm something he points out is a problem...) A recent point is the current Phoenix Suns...you look at the team and there's a ton of talent and yet they were ready to blow the whole roster apart a few days ago, only for Kerr to wise up it seems and decide that changing coach could produce immense change (I also disagree with Berri that "coaching doesn't matter") and I think it likely will, even before seeing the recent two explosive offensive games. Yet an adjusted +/- regression that treats the players as having uniform "state" across this season misses all kinds of nuance and detail before we even begin. Is Steve Nash post coaching change the same +/- value as Nash pre coaching change? Or how about Durant w/Carlesimo vs Brooks where he's been in different roles? Another example: Vince Carter has been a tremendous on/off player +5 in NJ '08-09 +12 in NJ '07-08 +9 in NJ '06-07 +15 in NJ '05-06 +11 in NJ '04-05 .... but -8 in TOR 04-05 http://www.82games.com/0405TOR.HTM +9 in TOR '03-04 Now it happened that Vince was unhappy and very vocal about wanting to be traded from Toronto in 04-05 right from the get go of the season...do ya think his mental state might have altered his usual 'constant' value some? Adjusted +/- completely misses a guy's emotional basis that changes, and completely misses physical issues...dealing with nagging injuries etc These points all serve to "invalidate" for me adjusted/regression +/- before we even get to bigger wholesale issues that arise and the "laugh test failures"...Dean's frescoball magical paper I think really frames the issue well for basketball and he was only looking at a two trait player...imagine what you have with basketball in terms of fit with loads of different skills you might classify and different roles for different positions, and different coaches asking different things of a player! The bigger question to me really is why people in the end think approaches that try to infer player contributions are better than simply tracking salient data to the point things are fairly self evident? It's safe to say that very soon data will be publicly available that will make much more transparent player individual defensive actions, shot creation, floor spacing, screen usage and screen setting, etc etc Why is there so much interest in turning to a mathematical construct for attempting to derive "statistical worth" when the raw material can be at hand to rather conclusively answer questions like "how good a help defender is Chris Paul?" or "how often does Paul attempting a steal hurt his team (vs the value of succeeding in the steal)?" or how about "on possessions where Devin Harris goes to the rim (and winds up in a heap under the basket), what is the change to his team's defensive pts/poss allowed coming back the other way?" To my mind there are going to be far more interesting stats to look at than regression outputs in the coming years... [I should also mention that while 82games has the unfortunately named 'roland rating' I wouldn't put much stock in that either...]
Back to top

Ryan J. Parker



Joined: 23 Mar 2007
Posts: 706
Location: Raleigh, NC
Posted: Thu Feb 19, 2009 1:08 pm Post subject:

Quote:
It's safe to say that very soon data will be publicly available that will make much more transparent player individual defensive actions, shot creation, floor spacing, screen usage and screen setting, etc
I figure the only guys that have this is Synergy, and they aren't (yet at least) just giving it away... I tried ripping some of it to no avail. I hope they have a change of heart. _________________I am a basketball geek.
Back to top

Neil Paine



Joined: 13 Oct 2005
Posts: 774
Location: Atlanta, GA
Posted: Thu Feb 19, 2009 1:25 pm Post subject:

Great posts, Roland & Dean. I could not agree more. There is no such thing as constant value -- as you mention, the same player can have radically different values to different teams, in different systems. "Fit" is hugely important, and we all but ignore that when we distill everything into a single, supposedly "context-free" number. I don't think any evaluatory method is going to be that panacea until there is some mechanism by which you can effectively predict the performance of any 5-man unit ahead of time (including a 5-man unit of all low-usage or all high-usage players).
Back to top

Mountain



Joined: 13 Mar 2007
Posts: 1527
Posted: Thu Feb 19, 2009 2:48 pm Post subject:

davis21wylie2121 wrote:
I don't think any evaluatory method is going to be that panacea until there is some mechanism by which you can effectively predict the performance of any 5-man unit ahead of time .
And my bet it is going to the adjusted approach fully applied to lineups that will get us closest. Players aren't constant in a range of 100+ lineups but lineups are lineups. Now opponents and style of play and schedule consideration vary but the 5 man lineup is as much stability as you'll ever get in basketball. Right now raw lineup data is what all but a few or one team sees on a current basis. Even that can be fairly helpful. I think taking a big minute success lineup and making it even bigger is the simplest available predictor of future lineup performance, short of having lineup adjusted, hence my advocacy for riding the winners you know pretty well even harder than usual like the Celtics do or the championship Pistons did, over all these spur of the moment scattered minute small lineups that all teams seem to have too much of in my opinion. Use the regular season to understand a few dozen lineups as well as you can. Pure adjusted in simplest form is not perfect and that is often noted by the authors, but it is important to acknowledge that we are generally much better off with it than without it. It recognizes most of the very best. It identifies many of the suspected bad wheels in a way they we didn't have before. It probably sorts most in the middle fairly well. 60-70% of the ratings are probably pretty close to what it estimates. Yes the outliers vex and can serve to push the method forward. We should give it time to move into its second phase and various forms including most importantly lineup adjusted. Multi-year and offense/defense has its place but it is probably not the end of the line. I think we will gain with publicly available statistical/pure blends, adjusted against playoff level teams and other style characterizations, adjusted in just the playoffs, adjusted player pair or triplets (representing cores that most lineups revolve around), adjusted lineups and multi-level cuts like adjusted lineup / against playoff level opponents / offense-defense split and maybe even that take to multi-season (at least this and last season and the previous playoff or two).Last edited by Mountain on Thu Feb 19, 2009 3:24 pm; edited 2 times in total
Back to top

schtevie



Joined: 18 Apr 2005
Posts: 397
Posted: Thu Feb 19, 2009 2:53 pm Post subject:

Dean, My strong sense is that you are being too agnostic by half. A player rating may be a single number purporting to represent "overall" value (and actual context, of course) but it is not necessarily boring, in the sense that there are important issues to which a single number can provide interesting and relevant insight (assuming that it is accurate, of course). A case in point to which I have recently referred is the value of Tony Allen. There are a lot of clever people out there who seem to feel that he is a bum. I cannot recall reading any published opinion (though I haven't searched for any, to be honest) that considers him to be a really good player. And his relatively low salary would seem to bear out this conventional wisdom. There is, however, a metric which is praiseworthy in its construction, though not without some flaws, that identifies him as a really good player. Particularly on the defensive end. Assuming this rating - this notionally boring number - is accurate, how would knowledge of this fact, by one or by all, not "accomplish a single thing". More generally, what is the bigger, more useful, truth that you feel is beyond reach? If your point is that there are more interesting questions to ask and deeper truths available that can only be answered by peeling back the layers of the onion, well, OK. But that is a separate issue.
Back to top

Mountain



Joined: 13 Mar 2007
Posts: 1527
Posted: Thu Feb 19, 2009 3:20 pm Post subject:

T Allen is a player pure +/- has never given a negative to. Generally nice positive, one neutral , one wildly high. I believe in Boston, in the role he has been given that he, on average, doesn't hurt and actually helps some and gets too much flak. But there are probably things he tries and shouldn't from time to time and good and weak lineups. So the pure adjusted is legitimizing on the roster but not the end of the management. There are teams I would have urged pursue T Allen now or later and heavily supported by the value measurement from pure adjusted.
Back to top

Kevin Pelton
Site Admin


Joined: 30 Dec 2004
Posts: 975
Location: Seattle
Posted: Thu Feb 19, 2009 3:57 pm Post subject:

schtevie wrote:
Assuming this rating - this notionally boring number - is accurate, how would knowledge of this fact, by one or by all, not "accomplish a single thing".
But that is precisely the issue! What evidence exists to justify the assumption that adjusted-plus minus is right and everything else is wrong? Are we to go strictly by the success of last year's Celtics?
Back to top

basketballvalue



Joined: 07 Mar 2006
Posts: 204
Posted: Thu Feb 19, 2009 4:11 pm Post subject:

HoopStudies wrote:
A player rating is compressing a lot of facts about a player (and his context) into a single solitary boring number. Given that the techniques are North-South-East-West of each other, there is no way that one number - any of them - can adequately tell the bigger more useful truth.
Roland_Beech wrote:
So I have the ironic position of being someone who has published all kinds of articles by adjusted +/- advocates (Dan, Dave, Aaron, Steve), and yet I think the approach is fundamentally flawed and while not worthless, certainly something to be very skeptical of. My thing is that I don't believe players have 'constant +/- value' which in the end is what the regressions are trying to do. I am in Dean's camp which is that "fit" is all important and that the whole notion of constant value is ridiculous.
Good posts everybody, I'm enjoying the thread even though people seem to be getting pretty fired up. We're all friends, right? I completely agree with your point, Dean, and have tried to emphasize that in the use of +/-. As far as your post, Roland, I also agree that people don't have a constant +/- value that just moves from team to team when player's change teams. Would you say the same thing about most other stats out there except perhaps FT% and a few others? Certainly I would think the constant value comment applies to PER and many more (including Roland Rating :]). If not, I'd really be intrigued to hear more of your thoughts about that. Thanks, Aaron PS I think it's pretty clear, but I hope people haven't confused the fact that just because I'm calculating and publishing +/- that I'm it's biggest proponent and think we should ignore all other information. It's just where I felt I could make my biggest contribution to the field when I started. I always consider other metrics when evaluating individual players._________________www.basketballvalue.com Follow on Twitter
Back to top

HoopStudies



Joined: 30 Dec 2004
Posts: 705
Location: Near Philadelphia, PA
Posted: Thu Feb 19, 2009 6:40 pm Post subject:

schtevie wrote:
Dean, My strong sense is that you are being too agnostic by half. A player rating may be a single number purporting to represent "overall" value (and actual context, of course) but it is not necessarily boring, in the sense that there are important issues to which a single number can provide interesting and relevant insight (assuming that it is accurate, of course).
So you're saying that I was right except for the "boring" part. I can live with that. I do admittedly love numbers, so maybe I hurt their feelings by calling them boring. I'm not saying that a summary rating has no value, just far less than we place on it by believing them so strongly (speaking of all metrics). schtevie wrote:
More generally, what is the bigger, more useful, truth that you feel is beyond reach?
Answering this the right way is one step towards getting a job doing it._________________Dean Oliver Author, Basketball on Paper The postings are my own & don't necess represent positions, strategies or opinions of employers.
Back to top

mtamada



Joined: 28 Jan 2005
Posts: 375
Posted: Thu Feb 19, 2009 6:54 pm Post subject:

davis21wylie2121 wrote:
I don't think any evaluatory method is going to be that panacea until there is some mechanism by which you can effectively predict the performance of any 5-man unit ahead of time (including a 5-man unit of all low-usage or all high-usage players).
That sets the bar perhaps a tad too high, but yup all along that is what I have said is the Holy Grail of basketball stats. The reason why I say it might be too demanding is that in practical terms (both for making useful predictions, and for having worthwhile data to base estimates upon), we will not often see those extreme hypothetical lineups that you describe, of 5 high-usage players, or 5 cloned Ben Wallaces, to use another example that has been mentioned. But aside from that, right on. Even avoiding those extreme situations, there are difficulties including: agreeing upon what an "effective" prediction is. E.g. is it even possible to take into account not just a player's role or fit with teammates, but the coaching style and decisions. Dick Motta seemingly underused both Dale Ellis and Detlef Schrempf with the Mavs. Bill Simmons has pointed out that D'Antoni's SSOL strategy, whatever else its merits, may be inflating the stats of the players who played in it: Nash, Marion, likely Stoudemire, etc. I.e. it may be an impossible task to predict the performance of a 5-man unit, if coaching strategies and decisions have significant impacts on what those 5 players do. Oh, one game where we do see 5 high-usage players on the floor at once: the All-Star game. But that game is so different from others, that I would not expect even the best stats to have much predictive value. (Might be an interesting test: maybe I'm wrong. Maybe there is a wonderful consistency to players' abilities as measured by whatever we think the best measure is. Then All-star game stats could be used to prove it -- a natural experiment! My strong hunch though is that all-star games are such a different environment that performance in those games will be difficult to predict effecitvely.) I guess the Dream Team, and Redeem Team, and other Olympic squads would be other examples of 5 high-usage players on the court simultaneously. The US teams' penchants for either destroying the opposition, or falling flat on their face (6th place in the 2002 World Championships and eking out a bronze in 2004) however suggests again that predictability is probably low.
Back to top

mtamada



Joined: 28 Jan 2005
Posts: 375
Posted: Thu Feb 19, 2009 7:16 pm Post subject:


Top
 Profile  
 
PostPosted: Fri Apr 15, 2011 5:51 pm 
Offline

Joined: Thu Apr 14, 2011 11:10 pm
Posts: 2548
mtamada wrote:
suggests again that predictability is probably low.
I should clarify that I think predictability is low for All-star games and probably Olympic games. For NBA teams, predictability is probably higher ... especially if the same coach (and generally similar coaching strategies) are employed from this season to the next.
Back to top



Page 8

Author Message Ryan J. Parker



Joined: 23 Mar 2007
Posts: 711
Location: Raleigh, NC
Posted: Thu Feb 19, 2009 7:28 pm Post subject:

I believe it's defeatist to worry about what a lineup of 5 Wallace-like players would look like. No one really wants the answer to that, as it's really not practical. That said, if we examine the performance of teams that play small ball, an uptempo pace, a slow pace, etc. we should be able to determine how this sort of coaching affects the players and hopefully parametrize it in some form or another. The common theme is that regardless of what we do or the data we have access to, we're simply trying to approximate reality. At first these approximations are probably going to suck. But overtime hopefully we will be able to refine them into something that makes good basketball sense and can help aid the decision making process. Either way they will never be a "perfect" model of reality. No model is._________________I am a basketball geek.
Back to top

schtevie



Joined: 18 Apr 2005
Posts: 414
Posted: Thu Feb 19, 2009 11:37 pm Post subject:

Dean, I think get the new religion. So, say a small forward for the three years preceding his possible contract extension produces APMs of 1.27, -0.49, and 0.86 (something below the positional average of about 1.0). Never mind that. I still shouldn't have confidence that this player is not worth 5 years and $80 million (where after he produces -5.08 and 1.61 - as an on-going two year average). There indeed must be a bigger, more useful truth that is beyond reach. And it is a job to answer this the right way.
Back to top

Neil Paine



Joined: 13 Oct 2005
Posts: 774
Location: Atlanta, GA
Posted: Fri Feb 20, 2009 12:47 am Post subject:

Dude, are you seriously taking a potshot at Dean because Mark Warkentien extended Carmelo Anthony's contract almost 3 years ago, coming off a season in which he scored 27 PPG on 48% shooting as a 21-year-old?
Back to top

Mountain



Joined: 13 Mar 2007
Posts: 1527
Posted: Fri Feb 20, 2009 3:30 am Post subject:

Yeah, metrics can sorta evolve into religions, especially in these public discussions,,, as opposed to how you treat them in personally directed evaluations / work plans. I think most people snap out of it enough see them as just analysis tools. '...there is no way that one number - any of them - can adequately tell the bigger more useful truth." That's why you, in part, turn adjusted into a series of products. I laid out how adjusted can so much richer with splits. As a summary tool lineup adjusted seems a darn fine summary tool for capturing the bigger more useful truth though. Study the entire production unit. Have you produced or tasked it for the Nuggets? Or used Eli's first rough cut? If not, how do you know how far it falls from telling the bigger more useful truth"? If not, are you just a priori convinced that smaller projects combined yield more or know that they are quicker / more likely to be applied for gain in your organization? Saying there is no Holy Grail has it is place...but then you get back to finding as much meaning as you can. As big and fine a chalice as you can. A lot of shot glasses can fuel the analytic review too but I'll take the product of a big cask or a big gulp and put it in the middle. A responsive one-liner: "Integrity is wholeness." Redundant? Yeah, on purpose to make its point. I'd never completely give up including that macro-perspective. You end up trying to cobble it back together from the dozens of micro-metrics but they never add up neatly to the whole. Pixelation and pretty grainy at that. And how is adjusted (split to offense and defense and taken together) at player or lineup level philosophically different than "offensive rating" or "defensive rating" or point differential? Do you still use these or only look at parts? 4 Factors tell useful / necessary details of the "truth" but don't you still put them together? Are you opposed to synthesis by numbers, not as the final score but simply as an organized way to be sure to include a lot of things, about the way you think it should be included to get most of the way there then philosophize, trade-off and guess and decide? To do the synthesis all in your head or in words... I am not sure that is "better". But whatever way you want to do it. I like to see the pieces and best representations of the whole side by side, in numeric form as much as you have and feel good about and bounce between them.Last edited by Mountain on Wed Oct 21, 2009 1:46 pm; edited 2 times in total
Back to top

Ilardi



Joined: 15 May 2008
Posts: 265
Location: Lawrence, KS
Posted: Fri Feb 20, 2009 12:00 pm Post subject:

Since I had the temerity to start this spirited thread - and since I'm in the apparent minority of APBRistas that regards adjusted plus-minus (APM) as an enormously useful evaluative tool - I feel some responsibility to address the many cavils and critiques that appeared on the thread yesterday. Unfortunately, I don't have the time to get to them all right now (that's not a dodge; I'm just on a tight deadline to return copy edits to a publisher). But I'll do my best to respond to some of the main issues in brief, and will be happy to weigh back in as soon as I can next week if anyone's still interested in this discussion by then . . . I guess the best place to start is with addressing Roland's points, since he's been kind enough to publish a few of my articles on 82games. He begins: Quote:
So I have the ironic position of being someone who has published all kinds of articles by adjusted +/- advocates (Dan, Dave, Aaron, Steve), and yet I think the approach is fundamentally flawed and while not worthless, certainly something to be very skeptical of. My thing is that I don't believe players have 'constant +/- value' which in the end is what the regressions are trying to do. I am in Dean's camp which is that "fit" is all important and that the whole notion of constant value is ridiculous. Players do have capabilities, tendencies, traits, etc but a player's contribution to team success is governed by certainly the specific teammates he plays with (and opponents he is asked to match up with), the coaches, the coaching schemes, the role he is asked to fill, the current physical condition/injury status, the current mental/ emotional state, contract status etc A somewhat preposterous way to look at constant value is to imagine a 'cloned' team...if you cloned Ben Wallace and stuck five of him on the floor at the same time I would argue you would have the worst NBA team in history...yet if you surround him with four well fitted teammates, with a coach (Larry Brown) whose schemes and motivational tactics worked, at a time when Ben was healthy and perhaps feeling a little under-appreciated ...you can win a championship with him. So where is the consistent impact? (Dave Lewin's article had Ben as the 3rd best player in the league...Rip Hamilton as the WORST in 05-06 http://www.82games.com/lewin2.htm something he points out is a problem...) A recent point is the current Phoenix Suns...you look at the team and there's a ton of talent and yet they were ready to blow the whole roster apart a few days ago, only for Kerr to wise up it seems and decide that changing coach could produce immense change (I also disagree with Berri that "coaching doesn't matter") and I think it likely will, even before seeing the recent two explosive offensive games. Yet an adjusted +/- regression that treats the players as having uniform "state" across this season misses all kinds of nuance and detail before we even begin. Is Steve Nash post coaching change the same +/- value as Nash pre coaching change?
Your main objection here, Roland, seems to be the premise of constant value of players across various lineups - a premise which you apparently regard as ridiculous, dismissing it as invalid a priori. But ultimately this is an empirical question, is it not? That is, for each player, there exists some coefficient of constancy that ranges from 0 (where context is everything, such that there are absolutely no regularities of impact across the player's roles or lineups) to 100 (where context means absolutely nothing: the straw man position apparently attributed to me and other APM enthusiasts). Surely you're not arguing that the coefficient has a value of 0? If so, please consider the answer to your above question about Steve Nash's value playing under Porter vs. D'Antoni-esque systems. Presumably, you asked the question to drive home the "obvious" fact that Nash would be much more valuable (with commensurately higher APM) in the D'Antoni-esque context? Well, last year under D'Antoni Nash had an estimated offensive APM of 10.01; this year under Porter (through 2/3) he has an offensive APM of 10.20. Last year his estimated defensive APM was -3.24; this year it's -1.86. Now, even though the APM analytic framework generates estimates of player main effects that assume a high degree of cross-context constancy, I've pointed out multiple times (on this listserve and in articles on 82games), that the APM model also permits empirical testing of player-by-player interaction effects - the very contextual ("fit") effects that have been raised as somehow fatal to the APM metric. In other words, APM has no problem with violations of the player constancy assumption: they merely reflect interaction effects. Your hypothetical "Ben Wallace x5" lineup would instantiate one such 5-way interaction, and I have no doubt that were such a lineup to be put on the court, the main effect of Ben Wallace on offense (i.e., his offensive APM) would be shown to interact significantly with that of his 4 Wallace-esque teammates (the same would doubtless also hold true for his defensive APM, but I'm guessing it would be to a lesser extent.) Quote:
Another example: Vince Carter has been a tremendous on/off player +5 in NJ '08-09 +12 in NJ '07-08 +9 in NJ '06-07 +15 in NJ '05-06 +11 in NJ '04-05 .... but -8 in TOR 04-05 http://www.82games.com/0405TOR.HTM +9 in TOR '03-04 Now it happened that Vince was unhappy and very vocal about wanting to be traded from Toronto in 04-05 right from the get go of the season...do ya think his mental state might have altered his usual 'constant' value some? Adjusted +/- completely misses a guy's emotional basis that changes, and completely misses physical issues...dealing with nagging injuries etc These points all serve to "invalidate" for me adjusted/regression +/-
I hope you can see the irony here: by showing how Carter's APM metric did a superb job of detecting his poor play in 04-05 (doubtless motivated by his desire to be traded), you've provided a compelling argument for the validity of the APM metric. The only thing that's potentially "invalidating" in your example would be if someone were to foolishly look only at Carter's 04-05 APM to the exclusion of all other seasons. As I've pointed out before, APM simply reflects what a player contributed to the team's bottom line for the period in question, not what he might have contributed had circumstances proven radically different. That interpretive caveat does not invalidate the metric; indeed, I believe it would have to be applied to any metric. Quote:
These points all serve to "invalidate" for me adjusted/regression +/- before we even get to bigger wholesale issues that arise and the "laugh test failures"
Much has been made in recent days of the apparent failure of APM to pass the laugh test vis-a-vis Chris Paul in 07-08, as it rated him a defensive liability for the season in question (for the record, as I've noted several times now, it still had him a top-25 rated player overall that season). Now, I'll certainly admit, if a metric yields myriad counter-intuitive, laugh-out-loud findings, at some point any reasonable person will conclude that the metric has to be deeply flawed. But that's not what I'm hearing about APM. In fact, the preponderance of player offensive and defensive ratings easily pass the laugh test - something I find pretty remarkable for a metric that doesn't use a single boxscore stat as an input. Instead, it appears to me that many here have been fixating on a single case - cherry picking the CP3 07-8 defensive APM number (I don't hear anyone complaining about the ridiculousness of his off-the-scale offensive APM number) as the single instance failure that somehow invalidates the entire enterprise, and ignoring the panoply of numbers that possess high face validity. (For what it's worth, last August I sat down with the brain trust of one of the NBA's most highly regarded front offices, and they told me the APM offensive and defensive numbers largely matched their own perceptions of player performance leaguewide for the period in question.) Of course, it could be pointed out that I was the one who raised the CP3 issue in this thread, so I should have no grounds for complaint! Indeed, I raised the issue in good faith, precisely because I'm interested in understanding the conundrum, and I assumed it might yield an enlightening discussion capable of clearing up the mystery of Paul's much-improved defensive rating from 07-08 to 08-09. And I believe it actually did so (I'll explain anon). But please note: no one on this thread has disputed the fact that the 07-08 Hornets were substantially less efficient defensively with CP3 on court, nor the fact that the 08-09 Hornets exhibit the opposite pattern, being much more efficient defensively with CP3 on-court, nor the fact that in Paul's on-court minutes, the 08-09 Hornets are more efficient defensively than the 07-08 team, despite Chandler's deteriorating performance. I have proferred CP3's improved defensive impact (suggested by his APM) as one plausible explanation for the above set of undisputed observations. I have yet to hear a single coherent or equally plausible alternative hypothesis. (Seriously, did I miss it?) Instead, I've heard an enlightening suggestion that CP3 may indeed have been less efficient last season in his frequent lineups (40% of his minutes) paired with short teammates in the backcourt (Pargo and Jackson). In other words, there's a plausible interaction effect that may moderate the main effect - and it's one that can be tested empirically within the APM framework. How, exactly, does any of this invalidate the APM metric? Now, I will confess to an occasional carelessness in language on a related thread last year that perhaps contributed to some of the ensuing difficulty. Specifically, by not adding enough nuance - by not emphasizing that CP3's poor 07-08 defensive number was simply a reflection of his overall impact on defense that season, and not (necessarily) an indicator of his being a defensive liability across all contexts - I inadvertently invited some unnecessary derision of the metric. Suffice it to say: lesson learned. Quote:
The bigger question to me really is why people in the end think approaches that try to infer player contributions are better than simply tracking salient data to the point things are fairly self evident? It's safe to say that very soon data will be publicly available that will make much more transparent player individual defensive actions, shot creation, floor spacing, screen usage and screen setting, etc etc Why is there so much interest in turning to a mathematical construct for attempting to derive "statistical worth" when the raw material can be at hand to rather conclusively answer questions like "how good a help defender is Chris Paul?" or "how often does Paul attempting a steal hurt his team (vs the value of succeeding in the steal)?" or how about "on possessions where Devin Harris goes to the rim (and winds up in a heap under the basket), what is the change to his team's defensive pts/poss allowed coming back the other way?" To my mind there are going to be far more interesting stats to look at than regression outputs in the coming years... [I should also mention that while 82games has the unfortunately named 'roland rating' I wouldn't put much stock in that either...]
I don't see this as a dichotomous, either-or prospect, Roland. In any domain of inquiry, there are multiple intersecting levels of analysis, and they reciprocally inform one another. As you know, I'm a mood disorders researcher, so when I want to understand something like depression, I'm going to look at an array of variables ranging from molecular to neurological to cognitive to behavioral to social. Sometimes it's important to pan out and take a wide, sweeping look at things - e.g., at an epidemiological level, we've seen a dramatic increase in depression prevalence over the past few decades. That's an important level of analysis in its own right, but of course we also want to look at what mediates this effect - what causes this increase - at lower-order levels of analysis. Likewise, APM provides a higher-order, panoramic view of player effects (not unlike an epidemiological prevalence number), but it tells us nothing about why those effects exist - for that, we need the sorts of lower-order data points you've mentioned (e.g., "opposing team's FG% following Player X's unsuccessful layup attempts"). Bottom line: APM and more molecular data points are not mutually exclusive forms of analysis. Rather, they are complementary, with the potential to reciprocally inform one another. I do not regard this process as boring - in fact, far from it. In closing, let me express my regret for the contentious tone that has sometimes characterized this thread. I enjoy the lively exchange of ideas as much as the next guy, but I think it's important for things to remain civil and respectful. After all, we all share a common love for the game, and for the spirit of inquiry that guides our analysis. I've genuinely tried to approach this conversation in that spirit, but admit to an occasional expression of irritation at what I've perceived to be an unnecessarily edgy, disrespectful tone on the part of one of our members. It is my sincere wish, however, that our future exchanges can prove worthy of the bonds that unite all of us. I'll certainly strive to do my part. Peace, SI
Back to top

Neil Paine



Joined: 13 Oct 2005
Posts: 774
Location: Atlanta, GA
Posted: Fri Feb 20, 2009 12:27 pm Post subject:

If APM, using interaction effects, can truly make a credible prediction for the point differential of a lineup of 5 Ben Wallace clones against an average NBA team, then I'd be sold. Because sans said interaction effects (which is sort of the only way the layperson has been able to make predictions using APM), a lineup of 5 prime Wallaces would supposedly be like +25 or +30 per 100 possessions -- which is insane, seeing as how they'd never get the ball over halfcourt, much less be able to shoot or really do anything offensively. But if interaction effects can take this into account and reflect that Wallace and his 4 clones, while 5 great role players for championship teams, would get destroyed on the court by themselves (an absurd example, to be sure, but a necessary thought experiment for any system such as this), then it would greatly increase my faith in APM.
Back to top

Westy



Joined: 15 Nov 2007
Posts: 15
Location: Chicago, IL
Posted: Fri Feb 20, 2009 12:40 pm Post subject:

Great post Steve!
Back to top

HoopStudies



Joined: 30 Dec 2004
Posts: 706
Location: Near Philadelphia, PA
Posted: Fri Feb 20, 2009 1:34 pm Post subject:

Ilardi wrote:
... Sometimes it's important to pan out and take a wide, sweeping look at things - e.g., at an epidemiological level, we've seen a dramatic increase in depression prevalence over the past few decades. That's an important level of analysis in its own right, but of course we also want to look at what mediates this effect - what causes this increase - at lower-order levels of analysis. Likewise, APM provides a higher-order, panoramic view of player effects (not unlike an epidemiological prevalence number), but it tells us nothing about why those effects exist - for that, we need the sorts of lower-order data points you've mentioned (e.g., "opposing team's FG% following Player X's unsuccessful layup attempts"). Bottom line: APM and more molecular data points are not mutually exclusive forms of analysis. Rather, they are complementary, with the potential to reciprocally inform one another. I do not regard this process as boring - in fact, far from it.
This is more useful conversation. I think Steve is saying that APM cannot exist in a vacuum. It is noisy and its lack of ability to say why it comes up with what it comes up with forces use of other tools. Discussing the "molecular data points" that BobC was putting out there to try to understand Why is fair. Am I wrong, Steve?_________________Dean Oliver Author, Basketball on Paper The postings are my own & don't necess represent positions, strategies or opinions of employers.
Back to top

Ilardi



Joined: 15 May 2008
Posts: 265
Location: Lawrence, KS
Posted: Fri Feb 20, 2009 1:45 pm Post subject:

HoopStudies wrote:
Ilardi wrote:
... Sometimes it's important to pan out and take a wide, sweeping look at things - e.g., at an epidemiological level, we've seen a dramatic increase in depression prevalence over the past few decades. That's an important level of analysis in its own right, but of course we also want to look at what mediates this effect - what causes this increase - at lower-order levels of analysis. Likewise, APM provides a higher-order, panoramic view of player effects (not unlike an epidemiological prevalence number), but it tells us nothing about why those effects exist - for that, we need the sorts of lower-order data points you've mentioned (e.g., "opposing team's FG% following Player X's unsuccessful layup attempts"). Bottom line: APM and more molecular data points are not mutually exclusive forms of analysis. Rather, they are complementary, with the potential to reciprocally inform one another. I do not regard this process as boring - in fact, far from it.
This is more useful conversation. I think Steve is saying that APM cannot exist in a vacuum. It is noisy and its lack of ability to say why it comes up with what it comes up with forces use of other tools. Discussing the "molecular data points" that BobC was putting out there to try to understand Why is fair. Am I wrong, Steve?
Thanks for your thoughtful post, Dean. And, yes, I certainly agree with you on this.
Back to top

schtevie



Joined: 18 Apr 2005
Posts: 414
Posted: Fri Feb 20, 2009 2:36 pm Post subject:

I like a group hug as much as the next guy, but before we agree to agree, we should agree on what we never disagreed about. By my reading of this and my recollection of all previous versions of this conversation, there has never been any controversy surrounding the notion that "APM cannot exist in a vacuum" or that APM's "lack of ability to say why it comes up with what it comes up with forces use of other tools." Neither has there ever been an assertion of the constancy of APM results (absolutely or relatively, in comparison to other metrics). Nor can I recall there ever having been the claim that APM can accurately predict out of sample wackiness, such as what a team of Ben Wallaces, or Shaquille O'Neals or Allen Iversons would look like. And certainly, there has never been the claim that APM provides the answers to many questions relating to the day to day management and coaching of a basketball team. But I am sure that there are areas of disagreement that remain. Dean refers to the noisiness of APM estimates being a problem. I am curious if Steve really agrees with this. To help make the joint points that all rating systems are equal but that some are more equal than others AND that a single, boring statistic can be highly relevant, I offered up the case of Carmelo Anthony. Admittedly, this is perhaps clearer than the truth, in that he is a conspicuous outlier in terms of the gap between offensive-focused conventional wisdom and APM, but it speaks to the issue of noisiness as well. Important roster decisions are necessarily based upon past as well as anticipated performance. With respect to offering him a five year contract, the probable 1.5 standard error on his then-to-date 1ish APM was well within the bounds of useful and actionable information. More generally, if individual player estimates are indeed independent of one another, multi-year estimates with standard errors around 1.5 provide a non-noisy basis for contemplating the effect of multiple roster moves. My point is, that in terms of how APM has always been represented as being relevant, it can't be written off for the noisiness of its estimates. This was the basis for my saying that the Celtics had a good thing coming in 2008, and this is a basis for it being the preferred metric for similar types of analysis. What GMs do is important, right? Can we agree on that?
Back to top

Ilardi



Joined: 15 May 2008
Posts: 265
Location: Lawrence, KS
Posted: Fri Feb 20, 2009 2:55 pm Post subject:

Dean refers to the noisiness of APM estimates being a problem. I am curious if Steve really agrees with this. Yeah, I agree - it's definitely a problem, but one that can often be overcome to a large extent. Think of it this way . . . using 5 seasons' worth of data to reduce noise, it's possible to get the standard errors on weighted seasonal APM offensive/defensive ratings for high-minutes players down around 1.0. That means we can be confident that 68% of the time any single player value we've estimated has a true value that's +/- 1 point of the estimated rating itself, and 95% of the time it's +/- 2 points. The noise hasn't been eliminated: the level's just a lot lower than it used to be. Now, don't get me wrong: I'm happy to be able to get the confidence intervals that narrow, as this is what allows the metric to be quite useful in its present form. But I'll be the first to admit that there's still enough noise in the system to prevent us from making fine-grained distinctions - at a high level of confidence - between players whose estimated ratings fall within a point or two of one another. I think that's why Mountain and others are still interested in augmenting APM with statistical regressions ("statistical plus-minus"), as Dan R. showed that it can, in principle, bring the errors down even further. There's still ample room for improvement. P.S. Ok, now I've really spent way too much time here today (it's so much more enjoyable than editing and writing end notes), but I need to go become a hermit until next Wednesday.
Back to top

DLew



Joined: 13 Nov 2006
Posts: 224
Posted: Fri Feb 20, 2009 3:19 pm Post subject:

This has been a very interesting thread. As most of you know I am a proponent of Adjusted Plus-Minus. I have published articles on it, and generally feel that it is useful. Please allow me to explain why. To me, there are three main things that we want to know about a player: 1. How good has he been? 2. Why was he effective/ineffective? i.e. How did he achieve his level of performance? 3. How good is he going to be? The first question is always the starting point to me. You can't know what is going to happen until you know what has happened. And I feel that adjusted plus-minus represents the truth in this regard, although it comes with a lot of noise (at least in some formulations). The second question is not really addressed well by adjusted plus-minus, as Roland and many others have pointed out, and is probably more important. Answering the third question involves combining the information from the answers to the first two questions in order to understand what type of performance can be expected from a player as his situation changes because of coaching changes, teammate changes, or just getting older and more experienced. I think that Roland makes a good point about how the linear model of APM assumes that a player performs the same across the period being considered with the only deviations from this being due to random variation. If this assumption is untrue then the model is invalid. However, I believe that although player performance varies significantly depending on the way in which the player is used, that in general this variation is not so large that it renders the technique useless. Adjusted plus-minus is a blunt instrument, and should be regarded as far from perfect, but I think that most of the opposition to it stems from the idea that it is supposed to be perfect. It is not, and when compared to all other overall player rating systems (which, as I've said before are not the be all and end all of analysis) it is clearly superior when done properly. The arguments that are brought against adjusted plus-minus, like that player performance is context dependent can be applied to any overall player rating system, and therefore should not be part of this discussion. If we are going to debate its merits, we should consider it a given that it has the flaw of summarizing player performance in one number (or two if we want to split it by offense and defense). This is not open for debate. Instead intelligent debate will center on whether it is better or worse than other metrics that share this same property.
Back to top

schtevie



Joined: 18 Apr 2005
Posts: 414
Posted: Fri Feb 20, 2009 4:01 pm Post subject:

Steve - who is not there - my question to you about noise not being the problem some fear it to be had less to do with using APM for considering the value of an individual player and more to do with using APM as a tool for evaluating hypothetical rosters. If the player estimates are uncorrelated, the variance of the sum decreases with the increasing number of players considered.
Back to top

HoopStudies



Joined: 30 Dec 2004
Posts: 706
Location: Near Philadelphia, PA
Posted: Fri Feb 20, 2009 4:51 pm Post subject:

schtevie wrote:
My point is, that in terms of how APM has always been represented as being relevant, it can't be written off for the noisiness of its estimates. This was the basis for my saying that the Celtics had a good thing coming in 2008, and this is a basis for it being the preferred metric for similar types of analysis. What GMs do is important, right? Can we agree on that?
I agree that GMs are important. If you're saying that picking the Celtics to win the title in 2008 based on APM is proof that it is not noisy, I don't agree. I hope no one would believe one instance of an accurate prediction as proof. You're generally talking about making predictions as evidence of a method's value. There are flaws with that approach (I've been told). But even if it is accepted, you gotta do more than 1. Or even do retrodictions, where at least you know player minutes after a season but use the prior season's best estimate of per-minute performance (or, essentially equivalently, per-possession stats). Since you now have APM on the O side and D side, predict team O Rtgs and D Rtgs, not just for one team. And then compare to other methods. (There are important nuances in doing this, but I'll let you figure those out.)_________________Dean Oliver Author, Basketball on Paper The postings are my own & don't necess represent positions, strategies or opinions of employers.
Back to top

Mountain



Joined: 13 Mar 2007
Posts: 1527
Posted: Fri Feb 20, 2009 5:46 pm Post subject:

The top 4 playoff teams the last 2 years have been lead by top 3 players who were all positive on the initial multi-season adjusted study Steve and Aaron published, except the 2007 Cavs. The Cavs changed. Melo's APM is relevant for contention. But, it is not terrible- and must be better this season- and he can probably work in his specialist role in a big 3 or 4 where the others handle the make the team better stuff and that was attempted last season and this season, and right now looks better. Still I'd trade him for a good offer. Previously. Or this summer. But I'll let you figure that out.Last edited by Mountain on Fri Feb 20, 2009 11:01 pm; edited 4 times in total


Top
 Profile  
 
PostPosted: Fri Apr 15, 2011 5:52 pm 
Offline

Joined: Thu Apr 14, 2011 11:10 pm
Posts: 2548
Page 9

Author Message bchaikin



Joined: 27 Jan 2005
Posts: 690
Location: cleveland, ohio
Posted: Fri Feb 20, 2009 7:01 pm Post subject:

Adjusted plus-minus... when compared to all other overall player rating systems... is clearly superior when done properly. clearly? might you care to substantiate this?... i ask as aside from the adjusted +/- chris paul poor defender in 07-08 claims, i seem to remember these claims from 8/2008 in the "shane battier and bobby jackson" thread concerning brad miller and amare stoudemire, this one from your compatriot: Yes, Miller has consistently (for several years) been a high adj +/- guy; Stoudemire, not so much (poor defense, alas). Accordingly, I believe Phoenix would have been better last year with Miller playing in place of Stoudemire. and this one from you: I just want to clarify that I am not arguing that Miller is better than Stoudemire. I am not at liberty to argue that one way or the other. I was merely commenting that, based upon publicly available data, Miller has consistently been one of the 3-4 best centers in the league over the period for which adjusted plus-minus has been available. one says miller is better than stoudemire in 07-08, the other no. are you two getting different results for the same player(s) from your adjusted +/- methodologies?... and miller in the top 3-4 Cs?... amare stoudemire was voted all-NBA 2nd team in 07-08. i don't believe brad miller got even one single vote. plus i don't know what publicly available data you might be referring to, all the offensive stats i see for stoudemire for all intent and purpose dwarf those of miller for 07-08, and the defensive counterpart data for the C spot for when each was on the floor for their respective teams in 07-08 at 82games.com: http://www.82games.com/0708/07SAC16C.HTM http://www.82games.com/0708/07PHO15C.HTM seems to indicate high offensive opponent production when each played, inferring neither was a good defender...Last edited by bchaikin on Fri Feb 20, 2009 7:18 pm; edited 2 times in total
Back to top

Mountain



Joined: 13 Mar 2007
Posts: 1527
Posted: Fri Feb 20, 2009 7:17 pm Post subject:

Based on APM I understand and agree with the generally positive view of Miller. Brad versus Amare is a hypothetical and I don't want to dwell on it but I think Miller on the Suns would fit and work pretty well. Amare / Porter didn't work but Amare / 7 seconds did. But now he's out so back to other stuff for the time being. But long-term I don't know that you go Nash-Amare-7 secs further except as last gasp hope in the face of nothing better. I think it is probably time to blow the whole thing up and start over, if it is championship or nothing. Dumars did that but it was Kerr that should have. He just added the Big Cactus and changed the coach and that wasn't enough or right.
Back to top

gabefarkas



Joined: 31 Dec 2004
Posts: 1313
Location: Durham, NC
Posted: Fri Feb 20, 2009 8:47 pm Post subject:

schtevie wrote:
Important roster decisions are necessarily based upon past as well as anticipated performance. With respect to offering him a five year contract, the probable 1.5 standard error on his then-to-date 1ish APM was well within the bounds of useful and actionable information. More generally, if individual player estimates are indeed independent of one another, multi-year estimates with standard errors around 1.5 provide a non-noisy basis for contemplating the effect of multiple roster moves. My point is, that in terms of how APM has always been represented as being relevant, it can't be written off for the noisiness of its estimates. This was the basis for my saying that the Celtics had a good thing coming in 2008, and this is a basis for it being the preferred metric for similar types of analysis. What GMs do is important, right?
So if you were assembling a team from scratch, who would you reasonably select to be on it? I include "reasonably" to take into account salary and playing time (ie, the 12th man almost never sees floor time) considerations.
Back to top

Mountain



Joined: 13 Mar 2007
Posts: 1527
Posted: Fri Feb 20, 2009 11:06 pm Post subject:

DLew, I agree with your post.
Back to top

DLew



Joined: 13 Nov 2006
Posts: 224
Posted: Sat Feb 21, 2009 4:09 am Post subject:

Bob, clearly? might you care to substantiate this?... No, I have contractual obligations not to, and frankly I wouldn't care to if I was allowed. If you choose to make an effort to understand adjusted plus-minus then you'll likely come around, but I suspect you've already made up your mind about it. and miller in the top 3-4 Cs?... Again, I can't comment on this. However, I will say that questioning along these lines indicates that you still don't understand what we have been saying about the noise associated with adjusted plus-minus. Because of the large standard errors on the coefficient estimates in all the results that are publicly available it is pointless to engage in most player vs. player debates. This is not to say that you can't empirically test the validity of adjusted plus-minus using the publicly available numbers, you can, but not with a sample size of two players.
Back to top

schtevie



Joined: 18 Apr 2005
Posts: 413
Posted: Mon Feb 23, 2009 2:45 pm Post subject:

HoopStudies wrote:
schtevie wrote:
My point is, that in terms of how APM has always been represented as being relevant, it can't be written off for the noisiness of its estimates. This was the basis for my saying that the Celtics had a good thing coming in 2008, and this is a basis for it being the preferred metric for similar types of analysis. What GMs do is important, right? Can we agree on that?
I agree that GMs are important. If you're saying that picking the Celtics to win the title in 2008 based on APM is proof that it is not noisy, I don't agree. I hope no one would believe one instance of an accurate prediction as proof. You're generally talking about making predictions as evidence of a method's value. There are flaws with that approach (I've been told). But even if it is accepted, you gotta do more than 1. Or even do retrodictions, where at least you know player minutes after a season but use the prior season's best estimate of per-minute performance (or, essentially equivalently, per-possession stats). Since you now have APM on the O side and D side, predict team O Rtgs and D Rtgs, not just for one team. And then compare to other methods. (There are important nuances in doing this, but I'll let you figure those out.)
Dean, I am not sure that you are reading what I was trying to write. Picking the Cs to win the title in 2008 based on APM was motivated by the expectation that the sum of the individual APMs would only be slightly biased when aggregating to the team level. The confidence I had that the estimate would be reasonably precise was based on the expectation that in summing individual APMs a lot of the noise of individual estimates would be expected to drop by the wayside. The latter point is related to econometric theory and has nothing to do with empirical confirmation. As for there being flaws with the approach of using predictions or retrodictions as the metric of a method's value, what are these exactly and why are they particularly relevant to the exercise in question? Yes of course "you gotta do more than 1", but at the same time, all experiments are not equal. This example was not cherry-picked. It was perhaps the largest net transfer of apparent talent to one franchise in the history of the league - two future HOFs for prospects. (My apologies to the historians for any egregious oversights.) If one's preferred scheme for appraising such momentous events doesn't get this kind of story straight, this is a big problem, no? It would not surprise me much if for the bulk of more or less average teams, with no conspicuous roster changes, that a variety of techniques would yield largely similar results. And furthermore that comparing these measures would persuade no one of anything. It is the outliers where the action is. But you are right, "you gotta do more than 1". I would hope that I could persuade someone with the data at hand to address the issue conclusively. I see that Neil has some interesting results with statistical +/- on basketball reference. Perhaps he would like to take the ball and run with it. Or Steve, he clearly has a lot of free time. But for fun, I did look at another team from the same season that had another kind of outlier status, the Chicago Bulls. Though here, this was a team that woefully underperformed relative to expectations, and as I recall there were no exceptional injury-related issues to explain their fall from grace. What the exercise in retrodiction shows (again, individual APMs summed in proportion to minutes, as opposed to possessions, played; rookies assigned actual APMs; replacement players assigned -7) was that using '06-'07 data, the predicted team APM for '07-'08 was 0.68 as opposed to the observed -2.94 (courtesy of basketballvalue). So, the prediction would be for a roughly 42 win team, instead of a 52 to 55 win team, courtesy of Dave Berri and John Hollinger. So APM does OK, not great, though I think that when it comes to teams in turmoil and free-fall, stemming from unmet expectations, one needs to cut all ratings systems some slack. Performing the same exercise, but using previous, single-season data, the prediction changes dramatically. Using numbers from '05-'06 and '04-'05 the prediction is a team APM of 7.53 and 6.61. And one final point which may be of some interest. Looking at the particular members of the Bulls roster and trying to tell myself a tentative story of what was going on, there was one individual contribution that was particularly notable: the Roland-ridiculed estimate of Ben Wallace in '05-'06 where his APM was 17.92. Of course, the rules are the rules for these prediction/retrodiction games, but this fact did lead me to finally read the fine print in Steve's 82games article on the '06-'07 season, and it appears that to lower standard errors, he blended in weighted '05-'06 data. Taking out the back of my envelope, I created a modified, interpolated set of values for '06-'07. The effect of this was to correspondingly reduce Ben Wallace's '06-'07 APM (and alter every other team member's as well). The overall effect of such tampering was to offer a revised '06-'07-based retrodiction of the '07-'08 Chicago Bulls' team APM of.....-1.68. Such shenanigans however do oblige full-disclosure. Doing the same modification of the Celtics data implies that they should have done even better than I actually predicted. And maybe they should have. According to '07-'08 APM, Ray Allen was actually a big disappointment. Perhaps the real story is what could have been had his off-season surgery not hampered his performance. No need to tamper with conventional wisdom too much however.
Back to top

Mountain



Joined: 13 Mar 2007
Posts: 1527
Posted: Mon Feb 23, 2009 5:15 pm Post subject:

Unweighted the positional averages of all positions fell short of Neil's projected statistical plus minus, the most at SG (-3) and the least at PG (less than -1). Weighted by minutes PGs outperformed projection significantly and Centers outperformed too but by a bit more than half as much. The SG-SF-PF middle guys underperformed with PFs and SGs underperforming by a third more than the PGs out performed. Altogether, minutes weighted players underperformed the projection at a rate I would consider significant. This in spite of a team efficiency improvement? Something is going on or the projection method needs adjustment.
Back to top

Mountain



Joined: 13 Mar 2007
Posts: 1527
Posted: Mon Feb 23, 2009 5:47 pm Post subject:

Looking at top lineups (raw) and everything else I'd give the edge to the Cavs over their main rivals. To beat Cavs somebody is probably going to have at least 2 good games guarding / limiting James. Pierce, Ariza and Turkoglu all had one regular season so there is reason to hope it can be done.
Back to top

trk



Joined: 26 Feb 2009
Posts: 12
Posted: Sat Feb 28, 2009 3:17 am Post subject: The effect of Chris Paul’s backups on his defensive Adj +/-

On of the problems I have seen with both raw and adjusted +/- ratings is that the ratings are often better at showing how good a player is compared to other players on the same team who play the same position than it at showing how good a player is compared to the league as a whole. Lets say there is a starter Aaron and a backup Bob who play the same position on the same team. Either Aaron or Bob is almost always on the court, but very rarely are both of them on the court at the same time. Aaron is a lot better than Bob, so the team tends to do better when he is on the court. As long the starters aren’t all subbed into and out of the game at the same time (which would add a bunch of new problems for any +/- metrics), adjusted plus minus should be able to easily figure out that Aaron is better than Bob. However, even after you figure that out, you still don’t know how good Aaron and Bob really are compared to the rest of the league. It could be that Aaron is an elite player and Bob is an average player, or Aaron could be the average player and Bob could be an exceptionally bad player. Since you rarely see them both on the bench or both on the court at the same time, it is very difficult to determine their value in absolute terms rather than just in comparison to each other. I think this effect might explain most of Chris Paul’s dramatic improvement in defensive adjusted +/- compared with last year. In 2007-2008, the PG when Chris Paul wasn’t on the court was usually either Bobby Jackson or Jannero Pargo. This year most of the backup point guard minutes are going to Antonio Daniels, with some minutes also going to Devin Brown. Now, I’m not an expert on the Hornets, but what I have hear of Antonio Daniels is that he is a poor defender who is getting worse as he ages. A check of Daniels’ production by position data from 82games shows that he has an unusually poor opponent PER of 20.0 when he plays at point guard, further backing up the idea that he is a poor defender. Is it possible that Chris Paul didn’t suddenly jump from being a very poor defender to being one of the league’s best, but that it just looks that way because he is now being compared with Daniels and Brown rather than Jackson and Pargo? The Hornets are allowing 105.2 points per 100 possessions while Chris Paul is playing this year, which is just slightly better than the 106.6 points per 100 possessions they allowed last year. This year the Hornets allow 110.6 points per 100 possessions without Paul (meaning Daniels or Brown was the PG) vs. only 100.6 last year (when Jackson or Pargo was the PG). Rather than concluding that “everyone got a lot worse on defense except Chris Paul who got a lot better”, isn’t it more logical to conclude that Paul and most of the other players haven’t changed that much but Daniels & Brown are much worse at defending the PG position than Jackson & Pargo?
Back to top

Mountain



Joined: 13 Mar 2007
Posts: 1527
Posted: Sat Feb 28, 2009 4:04 pm Post subject:

I think what you say is a big part of the story but might not be the whole story.
Back to top

Django



Joined: 22 May 2008
Posts: 4
Posted: Sat Feb 28, 2009 9:22 pm Post subject:

The biggest cause for higher opponent scoring when Paul is off the floor would be turnovers.Paul has an assist to turnvover ratio around 3.8 to 1 while his teammates turn the ball over at a much higher rate,especially when he's not on the floor.In fact their basic gameplan is to keep the ball in Paul's hands,minimising turnovers to keep their opponents from getting easy baskets.Since they are an offensively challenged team with only two reliable scorers at this point it's critical for them to make their opponents work hard to score,because they themselves aren't well equipped to win high scoring games.
Back to top

Crow



Joined: 20 Jan 2009
Posts: 821
Posted: Wed Oct 21, 2009 2:41 pm Post subject:

My previous skepticism about Carmelo Anthony was based on past individual Adjusted +/- data to support . He turned that number around this regular season and post season. Apparently they finally got him in contexts where he worked well, on average. Finally had all the ingredients around him to make it work- Nene's defense and rebounding, Billup's leadership and shot making, etc. Maybe it stays there, maybe not. I'd guess it probably will stay better than before but not sure how elite it will stay. But individual ratings are only an ingredient and is not the real destination, which is lineup performance. On Adjusted +/- at the lineup level in the playoffs the Nuggets 2 most used lineups came out a bit negative and more importantly far worse than the next 5 (all with Chris Andersen- perhaps he should have been used more?). The regular season Adjusted +/- at the lineup level suggested Anthony, - Billups- Nene - Jones - Martin wasn't good or even positive and the playoff data said almost the exact same thing. Should that lineup have been de-emphasized ? I'd think so (and said it several times last season) given the much stronger performance of other lineups but instead it went from being used about 1 in 7 minutes in the regular season to being used twice as frequently in the playoffs. Seems like a big disappointment- or error- there that could have been avoided, at least in terms of not doubling its use. If you gave weight to Adjusted Lineup data. But to be fair and to note the complexity of the challenge- the regular season data said Anthony, - Billups- Nene - Smith - Martin was great but the playoff data said it was a bit below neutral. The Nuggets may have listened to the regular season data (or got to the same decision by other means) because they increased the use of this lineup from 1 in every 24 minutes to 1 in every 9. A good idea but not all good ideas work. Was there a point where they should have seen it wasn't working that well? Aaron's game by game evolving lineup Adjusted data was available. If you did look I think you might have been able to see these lineups were really providing edge - on this measure- and could perhaps shift to trying other things more that were working better. Trend chasing can often disappoint but not noticing or chasing them can be a problem too. Should they have been content to be slightly negative on these lineups? Were they trying to win with other lineups? It is an available strategy but didn't quite work against the very best. Particular match-ups matter- team, position and where the rubber meets the road lineup vs lineup. One lineup tracked in the playoffs nearly exactly to what it did regular season and the other was wildly different. You can throw up your hands and ignore it as not a reliable guide with too large errors or go to the splits (regular season data against playoff level teams or "similar" teams to a specific match-up) and dig hard and ask why and see if you can find things or maybe even find ideas about things that might be useful in combination with other information. It still comes down to interpretation and blending all the input though.
Back to top

Crow



Joined: 20 Jan 2009
Posts: 821
Posted: Fri Nov 27, 2009 11:39 pm Post subject: Re: The effect of Chris Paul’s backups on his defensive Adj

Warning: Old thread but seemed appropriate to me to put these comments here since they pull on previous comments in this thread: trk wrote:
On of the problems I have seen with both raw and adjusted +/- ratings is that the ratings are often better at showing how good a player is compared to other players on the same team who play the same position than it at showing how good a player is compared to the league as a whole. ... I think this effect might explain most of Chris Paul’s dramatic improvement in defensive adjusted +/- compared with last year. In 2007-2008, the PG when Chris Paul wasn’t on the court was usually either Bobby Jackson or Jannero Pargo. This year most of the backup point guard minutes are going to Antonio Daniels... Is it possible that Chris Paul didn’t suddenly jump from being a very poor defender to being one of the league’s best, but that it just looks that way because he is now being compared with Daniels and Brown rather than Jackson and Pargo? The Hornets are allowing 105.2 points per 100 possessions while Chris Paul is playing this year, which is just slightly better than the 106.6 points per 100 possessions they allowed last year. This year the Hornets allow 110.6 points per 100 possessions without Paul (meaning Daniels or Brown was the PG) vs. only 100.6 last year (when Jackson or Pargo was the PG). Rather than concluding that “everyone got a lot worse on defense except Chris Paul who got a lot better”, isn’t it more logical to conclude that Paul and most of the other players haven’t changed that much but Daniels & Brown are much worse at defending the PG position than Jackson & Pargo?
This season Paul without Chandler, the Scott firing. other stuff and backed up at PG by Collison and B. Brown: Paul on the court team Defensive Rating 115.3 Collison 110.6. B Brown 111.6 It is real early of course but wonder what kind of Defensive Adjusted rating Paul gets this season.Last edited by Crow on Sat Nov 28, 2009 2:51 pm; edited 3 times in total
Back to top

Crow



Joined: 20 Jan 2009
Posts: 821
Posted: Fri Nov 27, 2009 11:43 pm Post subject: Re: The effect of Chris Paul’s backups on his defensive Adj

3 of the 4 factors worse on than off including shot defense and forced turnovers. Paul has below average counterpart defensive data right now too.Last edited by Crow on Sat Nov 28, 2009 1:52 am; edited 1 time in total
Back to top

Crow



Joined: 20 Jan 2009
Posts: 821
Posted: Sat Nov 28, 2009 12:21 am Post subject:

DLew wrote earlier in this thread: "To me, there are three main things that we want to know about a player: 1. How good has he been? 2. Why was he effective/ineffective? i.e. How did he achieve his level of performance? 3. How good is he going to be? ... The second question is not really addressed well by adjusted plus-minus, as Roland and many others have pointed out..." Perhaps 4 Factors Adjusted will help with Question 2. And more attention to lineup Adjusted +/- and sub-unit Adjusted (e.g. player pairs). Are there top teams with analytic shops not using Adjusted or the full range of Adjusted versions and splits at all or hardly at all? It seems like a lot to give up or push to the periphery. Or do they use it but just not promote or acknowledge as highly as others? What are they are using instead beyond the known discrete action tracking tools (items in the boxscore or the additions that are easily identified whether they are available from PBP parsing, Synergy video or manual, insider-informed compilations- what was supposed to happen vs what did) or products built off them? Are they making better decisions mainly with these other tools than those also using Adjusted tools or using them a lot more? Paul is a contentious test case but I believe digging into test case can potentially help. Anthony is another. He looked good by Adjusted last season, far better than most of the time previously. Believe it or not- then or for thinking about the future? This season his early small sample Adjusted is back at -5. Complicated. I tend to think that further study of the role of lineups and sub-unit / player interactions might help improve explanatory power and perhaps predictive power beyond where it often gets left right now in public. Whether it be by Adjusted techniques and / or otherwise. By Adjusted lineups the simplest stories for contenders are being written in Dallas, Denver, Phoenix and with the Lakers. Whatever you take from that. None of these big believers in Adjusted? Different tools got them to the desired place as measured by lineup Adjusted? Virtually everywhere the Player level Adjusted is complicated right now though= to my eyes. Including Cleveland and Houston where there is also less consistently good lineup Adjusted estimates for top lineups. I am guessing these teams use Adjusted techniques the most? Issues related to more and more important newcomers hard to predict with Adjusted or fix quickly? Seems like last season was smoother sailing at the lineup level. What to make of the use or non-use of tools and the results by different levels of Adjusted or simply point differential or win %? I sometimes want to inject "What about the Lakers' approach" or more generally "What about systems / coaches?" how much does that explain by these top contenders (including Dallas, Denver, Phoenix) using Adjusted techniques heavily or not?


Page 10

Author Message Crow



Joined: 20 Jan 2009
Posts: 796
Posted: Sat Nov 28, 2009 3:26 pm Post subject:

There is so much that is known beyond presence on the court that might be useful in refining Pure Adjusted. Different from moving into the world of Statistical / Adjusted blends or hybrids. It could be tested. Maybe information such as who starts or closes games, or the average length of stint on the court or the frequency of getting pulled quickly. With all the statistical techniques out there I'd think that Pure Adjusted could be pushed much further towards accuracy. If assuming and reporting a constant valuation is a major stumbling block for acceptance you could 1) move to reporting a range or 2) maybe re-run the calculation in some fashion allowing the search for the best set of 3 values for players (higher, average and lower performance in that possession or stint) for explaining the league data instead of just one. But ultimately this is work for top statistical analysts.
Back to top

Crow



Joined: 20 Jan 2009
Posts: 796
Posted: Mon Nov 30, 2009 9:54 pm Post subject:

For what it is worth, I noticed that the average raw on/off for the best 50 on Adjusted this season is 68% of their average 1 year Adjusted +/-. The average raw on/off for the worst 50 on Adjusted this season is 46% of their average 1 year Adjusted +/-. The average raw on/off for the best 50 on Adjusted last season was 94% of their average 1 year Adjusted +/-. The average raw on/off for the worst 50 on Adjusted last season was 47% of their average 1 year Adjusted +/-.
Back to top


Display posts from previous:


Top
 Profile  
 
PostPosted: Thu Sep 08, 2011 10:11 am 
Offline

Joined: Fri Apr 15, 2011 12:02 am
Posts: 2371
Location: Asheville, NC
Ime Udoka played for Portland in 2006-07. They were 32-50 that year.
Their SRS (pt diff + SOS) was -3.78, so their players' plus-minus will sum to that.
Keep digging!


Top
 Profile  
 
PostPosted: Thu Sep 08, 2011 6:18 pm 
Offline

Joined: Thu Apr 14, 2011 10:41 pm
Posts: 815
Location: Hotlanta
Yeah, they could be onto something with this apm stuff.

_________________
The City: A Golden State Warriors-Centric NBA Blog
NBA WOWY: Find statistics of your favorite team with any arbitrary combination of players on or off the court.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 8 posts ] 

All times are UTC


Who is online

Users browsing this forum: Yahoo [Bot] and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group