The Flaws of Synergy for Defensive Rankings

[EvanZ]

Players like Bonner will also not become defensive liabilities that easily, because such players tend to reduce the turnover rate for the team, which means less fast break opportunities for the opponents, and they also are set to go back on defense quickly, instead of crashing the offensive board.

This is an interesting observation, as it is true that Bonner apparently reduces team turnover rate. Here is a list of the top 10 in terms of adjusted offensive turnover ratio this season:

Code: Select all

FULL	TEAM	OTOR
Louis Williams	PHI	-0.96
Matt Bonner	SAS	-0.84
Thaddeus Young	PHI	-0.80
Jodie Meeks	PHI	-0.79
Elton Brand	PHI	-0.59
Devin Harris	UTA	-0.58
Nate Robinson	GSW	-0.58
Blake Griffin	LAC	-0.54
Chris Paul	LAC	-0.52
Andre Iguodala	PHI	-0.52

(It's also interesting that Philly has 5 of the top 10. In fact, that team looks like a historic outlier in terms of having a lower TOR than the rest of the league. They were a full 2 %-points lower than the second best Clips.)

Anyway, maybe this also addresses Bob's questions as to how Bonner creates value. "Spreading the floor" and "lowering team turnover rates" are two things that aren't easily quantified with traditional metrics.

[mystic]

(It's also interesting that Philly has 5 of the top 10. In fact, that team looks like a historic outlier in terms of having a lower TOR than the rest of the league. They were a full 2 %-points lower than the second best Clips.)

Well, watch them play, it becomes clear why. They let their better ball handlers create their scoring opportunities. There is not much passing going on, it is either Holiday, Williams, Turner or Iguodala dribbling the ball on the perimeter and then either attacking or shooting the ball. If they are passing it inside, it is again up to Hawes or Brand to create or they getting it due to drive&dish. Really, as a Bulls fan I probably saw great defense by the Bulls, but also really stagnant offense in the last couple of days.
Overall that helps their defense to get set, because I have not the impression that their halfcourt defense is that great.

Also, one thing I saw is the low amount of offensive fouls. Not quite sure whether that is indeed true for all of their games, but somehow they rarely getting called for offensive fouls.

Anyway, maybe this also addresses Bob's questions as to how Bonner creates value. "Spreading the floor" and "lowering team turnover rates" are two things that aren't easily quantified with traditional metrics.

Indeed, we can hardly quantify that. And overall we have probably a lot of different small effects which are adding up. Maybe even in a non-linear fashion, which makes it pretty hard to understand that with our brain mainly working linear.

[YaoPau]

Those are legitimate questons. And that has indeed not much to do with sampling errors.

Let us take a look at Carter. Carter is big for a SG, he does not have problems to compete with any kind of other SG. That helps his defense, especially when guards are trying to post him up. Carter is one of the best in terms of defending the post-up. Carter has still long arms and is pretty quick on his feet. He can cover ground and that helps to defend shots at the perimeter. His opponents does not have an easy way to get by Carter, Carter stays in front of him and that forces more contested shots. Compare Carter and Bryant for example in terms of closing out on a 3pt shooter, Carter is better at that. It is also the case that Carter can defend the p&r pretty well. In fact the ball handler is more often in trouble. Overall Carter's defensive abilities and his size is causing a clear positive impact on the defensive end. Tougher shots for the opponents with a clearly lower conversation rate. That explains Carter's positive defensive RAPM. The value comes mainly from him being matched up with SG. When he plays SF, his quickness in comparison helps to limit drives to the basket. But he also doesn't have the size advantage anymore, which takes value away.

His value on offense comes from shooting, passing and from attacking the basket. His shooting range helps spreading the floor. Especially in comparison to Marion at SF, the Mavericks with Carter are becoming a much better offensive team while Carter plays the SF spot. The floor spreading is not just seen by clearly better 3pt shooting, but opens up the middle also for Nowitzki and makes it easier to create those midrange shots. Carter's passing helps to limit turnovers, provide better ball movement, which overall leads to a better team offensive efficiency.

Carter in 2009 had a negative RAPM ... either he got significantly better at age 35, or there's sampling error. He started in 2009 and in 2012, so I wouldn't expect playing situationally to be a factor, like it was proposed for Bonner.

Obviously Carter has ability, but at this point in his career there are dozens of other wings who can do similar things. Yes Carter rated well defending the post, but he only faced 31 post-up possessions all season, so I doubt that matters much. And wings routinely rate really well in post defense because their opponents usually aren't as skilled in that position. See guys like Carmelo, Joe Johnson, Lou Williams, Daequan Cook, Roddy Beaubois who aren't regarded as great defenders but had a better PostUp defense rating this year. P&R same thing ... faced just 59 possessions, and nobodies like Jason Kapono, Roger Mason, Mike Miller rated ahead of him. Carter's defensive numbers also have been pretty average in past years...

But I think the easier argument is that, while Carter has his strengths, so do Chris Paul, Kevin Garnett, Dwyane Wade, Pau Gasol, Dwight Howard, Kevin Durant, and Kobe Bryant ... and yet Carter is rated ahead of all of them. I can agree that Carter is still a positive contributor, but he doesn't have the track record of elite defense, and doesn't have the box score numbers to make me think he was a top 10 efficiency player this season while playing on a starting line.

RAPM is basically a proportional shrinkage of APM ... there's just under a .8 correlation between the two ratings when factoring in minutes played. 1-year APMs clearly have huge standard error issues, so there's no reason to expect standard error to not be a problem in RAPM. Fewer weird multicollinearity effects, but there will still be overestimated and underestimated players.

[mystic]

Carter in 2009 had a negative RAPM ... either he got significantly better at age 35, or there's sampling error. He started in 2009 and in 2012, so I wouldn't expect playing situationally to be a factor

What? Carter played a completely different role in 2009 and 2012. He had a much greater offensive load in 2009 and was asked to carry the Nets offensively. It is not even the amount of shots taken, but the time Carter had the ball in his hand in order to initate offense. He had clearly less focus on the defensive end. Such a thing is seen by many wing players.
Carter showed similar defensive strength in other seasons. Thus, saying Carter would have needed to get better at the age of 35 is not just weird, but clearly wrong.

I can agree that Carter is still a positive contributor, but he doesn't have the track record of elite defense, and doesn't have the box score numbers to make me think he was a top 10 efficiency player this season while playing on a starting line.

Defensive RAPM for Carter

2004: +1.1
2005: +1.1
2006: +2.4
2007: -0.1
2008: +1.1
2009: -1.1
2010: +1.7
2011: +1.2
2012: +1.9


Carter had pretty good years before, and when he was focussed defensively, he was also doing really good in defensive RAPM before.

RAPM is basically a proportional shrinkage of APM

No, it is not. How can you say that Ridge Regression is just a shrinkage of OLS?

1-year APMs clearly have huge standard error issues, so there's no reason to expect standard error to not be a problem in RAPM. Fewer weird multicollinearity effects, but there will still be overestimated and underestimated players.

Standard errors are making no sense, because their is a bias introduced by using the ridge regression. And while there will still be over- and underrating of players, multicollinearities and overfitting is basically eliminated by using ridge regression. The over- and underrating comes rather from the fact that players aren't consistent, and even within a year their performance level is shifting, either due to injuries, role changes or mental approaches (tanking at the end of the season, increased motivation towards the end in contract years, etc. pp.). Using regression in order to determine a somewhat intrinsic values goes hand in hand with the assumption that the players are constant in their performance level.


Oh, and even on a team level the boxscore can only explain 50% of the variation of the defense. There is just not enough numbers in the boxscore in order to get a more useful answer than using a coinflip to determine whether someone is good defensively or not. Take Carlos Boozer for example. His individual defensive rating (Oliver's approach) looks great in the last two seasons (99 and 95 respectively), while he is really a bad defender.

[Bobbofitos]

Take Carlos Boozer for example. His individual defensive rating (Oliver's approach) looks great in the last two seasons (99 and 95 respectively), while he is really a bad defender.

Booze was #2 in minutes played (over 60% of the Chicago Bulls' minutes this season)... The Bulls had the 2nd best DRtg this year. Is it really fair to say he is a really bad defender?

Looks to me like, although perhaps not a positive/leading to the end result, he objectively didn't really prevent the Bulls from having one of the best defenses in the NBA.

[mystic]

Booze was #2 in minutes played (over 60% of the Chicago Bulls' minutes this season)... The Bulls had the 2nd best DRtg this year. Is it really fair to say he is a really bad defender?

It is fair given the fact that the Bulls played nearly 11 points worse defensively when Boozer was on the court. Boozer has the worst difference between On and Off court DRtg.

Looks to me like, although perhaps not a positive/leading to the end result, he objectively didn't really prevent the Bulls from having one of the best defenses in the NBA.

Given the fact that the Bulls without Boozer on the court had a 91 DRtg, he objectively prevented the Bulls from having the best defense in the NBA.

[YaoPau]

Standard errors are making no sense, because their is a bias introduced by using the ridge regression. And while there will still be over- and underrating of players, multicollinearities and overfitting is basically eliminated by using ridge regression. The over- and underrating comes rather from the fact that players aren't consistent

Just because the standard errors aren't easily interpreted doesn't mean they aren't there. And just because a bias is introduced doesn't mean the variance is effectively driven to 0. Ridge is just used to trade some bias for a variance reduction ... either way, if you have large bias and little variance, or little bias and large variance, there's going to be inaccurate results, especially when you're trying to fit a simple 5 input vs 5 input linear model to something that obviously isn't that simple.

We're dealing with a small sample of data here, with inputs that are very hard to separate from one another. In regular APM, only about 20 of the 450ish NBA players have significant coefficients, just massive standard errors. All we're doing with ridge is introducing a constraint on the parameters... that will definitely help the occasional multicollinearity issues, but it's not going to make the model as accurate as you're interpreting it to be, nowhere close.

[mystic]

And just because a bias is introduced doesn't mean the sampling error is effectively driven to 0.

I don't think that the sampling error is such a big problem, especially when priors are used.

especially when you're trying to fit a simple 5 input vs 5 input linear model to something that obviously isn't that simple.

I agree, it is likely not a simple linear relationship, but we can in a lot of cases approximate that with a linear model. We can test the model by making predictions (or out of sample tests).

We're dealing with a small sample of data here

No, not really. We have a much bigger issue than the sample size. The fact that we have an ill-posed problem is more a concern. That shows up when we try to use APM in order to explain the wins for teams. That is really worse than using RAPM here instead.

that will definitely help the occasional multicollinearity issues, but it's not going to make the model as accurate as you're interpreting it to be, nowhere close.

How would you know how "accurate" I interpret the result? Sounds like a really weird statement given the fact that I did not say at any point how much weight I put into the numbers at all.

[YaoPau]

How would you know how "accurate" I interpret the result? Sounds like a really weird statement given the fact that I did not say at any point how much weight I put into the numbers at all.

why does vince carter rate so high in this version, or any version, of APM?...

why does matt bonner rate so high in this version, or any version, of APM?...

are these not legitimate questions to ask?...

Those are legitimate questons. And that has indeed not much to do with sampling errors.

You're taking Carter's and Bonner's RAPMs at face value.

[mystic]

You're taking Carter's and Bonner's RAPMs at face value.

Sorry, but you are running into a typical fallacy here by assuming something which was not said. I just agreed with EvanZ that this can't be explained by sampling errors, because we see similar numbers in other years as well. That does not mean that I take that as face value, not at all.

So, again, where did I say how accurate those numbers are? Actually, I even said that over- and underrating of players will likely happen just due to the nature of game (variance of the player performance) and the method (assume to be static performance level). Well, you might I understand my confusion with your statement?

[EvanZ]

I'll be posting these on my site soon. Here's the top 25 players this season with >2500 possessions:

A4PM: TOP 25 >2500 POSS

Code: Select all

FULL	TEAM	POSS12	A4PM	VARP
Dwyane Wade	MIA	2829	2.84	137.02
Tony Parker	SAS	3338	2.59	153.36
Danilo Gallinari	DEN	2563	2.35	111.54
LeBron James	MIA	3906	2.30	167.93
Chris Bosh	MIA	3358	2.22	141.81
Dwight Howard	ORL	3407	2.22	143.61
James Harden	OKC	3384	2.10	138.81
Elton Brand	PHI	2674	2.09	109.37
Blake Griffin	LAC	4177	2.09	170.81
Andre Miller	DEN	3280	2.09	134.08
Ryan Anderson	ORL	3015	2.04	121.85
Kevin Garnett	BOS	2924	2.01	117.14
Chris Paul	LAC	3790	1.98	150.80
Vince Carter	DAL	2699	1.90	105.30
Channing Frye	PHX	2928	1.87	113.33
Hedo Turkoglu	ORL	2633	1.81	100.33
Ronnie Brewer	CHI	2716	1.79	103.04
Tony Allen	MEM	2791	1.77	105.23
Dirk Nowitzki	DAL	3634	1.70	134.61
Al Harrington	DEN	3250	1.66	118.99
Caron Butler	LAC	3247	1.55	115.41
Russell Westbrook	OKC	4079	1.53	144.11
Thaddeus Young	PHI	2811	1.53	99.09
Rajon Rondo	BOS	3199	1.52	112.64
Louis Williams	PHI	2705	1.51	95.06

Here is the top 25 of players with 1000-2500 possessions:

A4PM TOP 25 1000<POSS<2500

Code: Select all

FULL	TEAM	POSS12	A4PM	VARP
Matt Bonner	SAS	2257	3.66	127.77
Manu Ginobili	SAS	1404	3.56	78.10
Beno Udrih	MIL	1835	3.47	100.31
Zach Randolph	MEM	1304	2.62	60.25
Taj Gibson	CHI	2155	2.57	98.41
Jeremy Lin	NYK	1447	2.21	60.94
Stephen Curry	GSW	1417	2.13	58.58
Avery Bradley	BOS	2119	2.05	85.90
Larry Sanders	MIL	1147	1.99	45.73
Derrick Rose	CHI	2473	1.98	98.30
Ekpe Udoh	MIL	2246	1.97	89.17
Tiago Splitter	SAS	2052	1.85	79.05
John Lucas	CHI	1180	1.83	45.23
Chauncey Billups	LAC	1072	1.79	40.60
Josh Howard	UTA	1720	1.56	61.27
Nick Collison	OKC	2251	1.53	79.43
J.R. Smith	NYK	1588	1.51	55.69
Jared Jeffries	NYK	1186	1.43	40.70
Steve Novak	NYK	1704	1.42	58.24
Grant Hill	PHX	2453	1.39	83.17
Vladimir Radmanovic	ATL	1123	1.33	37.41
Linas Kleiza	TOR	1122	1.33	37.31
Daequan Cook	OKC	1707	1.30	56.27
Ian Mahinmi	DAL	1935	1.28	63.46
Kris Humphries	NJN	1563	1.25	50.80

[greyberger]

While Matt Bonner was being discussed I wanted to chime in with my pet theories.

I'd start by looking at his on/off four factors impact and teammate profile. For example in 2012 Bonner was paired in the frontcourt with Splitter 53% of Bonner's total minutes, Duncan 29%, and Blair 12% with almost no overlap. The context here is Duncan and Splitter have solid +/- numbers while Dejuan Blair has trainwreck numbers. If Duncan and Splitter are indeed positive influences, I imagine Duncan helps the defense while Splitter is more of a help on offense (21% USG .65 TS%). Blair helps Bonner in that he's played so poorly and makes for a stark contrast and because he played against starters while the Bonner lineups went against deep bench lineups and mixed bench lineups.

So it's Bonner and Splitter or Bonner and Duncan - it's Bonner and a quality screen and roll guy. Add a player who can make good use of those screens, either Parker (48%), Ginobili (29%) or Neal (50% with overlap). The other two spots go to shooters, who each get to stand in a corner. Now you run screen and rolls with Bonner positioned for an angle three or involved in some way himself that allows him to draw his defender away from the real action.

Apparently anybody can make an open corner three. It's 22 feet and there's a big difference from the 24-foot angle threes in average efficiency and difficulty. It's probably going too far to say it's a skill that can be taught, but as far as shots go it seems to be within the skills of many NBA players. Danny Green and Kawhi Leonard developed this shot after moving to SA despite justifiable skepticism that they would become three-point threats. The companion point is not all three point shots and shooters are the same. Bonner's shooting versatility keeps his defender out there and gives the lineups a brutal simplicity - defend this SnR play, no help defense or we get a good look from three.

The Bonner lineups are exceptionally good at shooting margin (offense and defense), and good at turnover margin (by avoiding them) and FT margin (by drawing them and not fouling). Here's your on/off table for Bonner 2012.

Code: Select all

  	              OffRtg	DefRtg	NetRtg	OREB%	DREB%	REB%	ASTRatio	TORatio	eFG%	TS%
Totals 	         108.5 	100.6 	7.9 	25.1% 	76.0% 	51.1% 	17.9 	10.51 	52.9% 	56.2%
Bonner on court 	113.5 	100.1 	13.4 	22.8% 	76.5% 	50.5% 	18.0 	9.44 	55.4% 	58.5%
Bonner on bench 	105.0 	101.0 	4.1 	26.6% 	75.6% 	51.4% 	17.9 	11.25 	51.1% 	54.5%

Bonner played 1346 minutes out of a possible 3188, Blair started 62 games and played 1363. They only had 162 minutes on the court together, so Bonner without Blair or Blair without Bonner accounted for 75% of possible time for the Spurs. Blair just seems to be really bad or a really bad fit and I don't know how much action he'll see in the playoffs. If Tiago Splitter wasn't briefly injured in the Jazz series Blair would have been the fifth big in minutes, and by game four Dejuan got seven minutes to the other bigs' 20. You have to wonder if starting Blair all season was an attempt to craft him into a useful playoff piece or an unsuccessful attempt to build up his trade value. Diaw and Splitter are better for when the Bonner lineup isn't appropriate.

As for whether Bonner's numbers would be just as good in more minutes, I guess we'll just have to wait and see. SA will probably be increasing the minutes given over to these lineups in the playoffs and next year.

[YaoPau]

I think we're still talking separate things. I dunno, maybe not.

Evan was saying Bonner's RAPM isn't necessarily due to sampling error, which I'm fine with. "indeed not much to do with sampling errors" is a different thing entirely.

The over- and underrating comes rather from the fact that players aren't consistent, and even within a year their performance level is shifting, either due to injuries, role changes or mental approaches (tanking at the end of the season, increased motivation towards the end in contract years, etc. pp.).

You're talking about an issue with measurement accuracy, and that's somewhat separate from variance, which is a randomness in sampling issue. It sounds like you're assuming the sample size of 66 games is plenty big enough, meaning there is basically zero randomness and no variance. IMO most of the over and underrating in a non-prior APM comes from 66 games being way too small a sample, which produces large coefficient errors.

[YaoPau]

I like the thought grey. I'm trying to find another situation where two players usually sub in for each other, and one of them has produced really poor results, and it creates an odd-looking APM for the other guy.

Maybe Jameer Nelson and Chris Duhon this year in Orlando? Both guys have huge APMs despite unremarkable on/off numbers. Orlando experimented with Ishmael Smith and Larry Hughes at point guard and that was a disaster. Although RAPM saw through it for the most part it looks like.

[EvanZ]

While Matt Bonner was being discussed I wanted to chime in with my pet theories.

I'd start by looking at his on/off four factors impact and teammate profile. For example in 2012 Bonner was paired in the frontcourt with Splitter 53% of Bonner's total minutes, Duncan 29%, and Blair 12% with almost no overlap. The context here is Duncan and Splitter have solid +/- numbers while Dejuan Blair has trainwreck numbers. If Duncan and Splitter are indeed positive influences, I imagine Duncan helps the defense while Splitter is more of a help on offense (21% USG .65 TS%). Blair helps Bonner in that he's played so poorly and makes for a stark contrast and because he played against starters while the Bonner lineups went against deep bench lineups and mixed bench lineups.

You might be interested in my adjusted four factors +/- (A4PM):

http://thecity2.com/2012/02/21/new-play ... ctor-a4pm/