APBRmetrics

The discussion of the analysis of basketball through objective evidence, especially basketball statistics.
It is currently Mon Aug 21, 2017 4:32 am

All times are UTC




Post new topic Reply to topic  [ 35 posts ]  Go to page 1, 2, 3  Next
Author Message
PostPosted: Sat Oct 01, 2011 2:25 pm 
Offline

Joined: Sat Oct 01, 2011 2:22 pm
Posts: 18
We've written a new paper about chemistry in the NBA, using a novel framework we call "Skills Plus Minus." This framework lets us evaluate the synergies of players and teams.

It also lets us find mutually beneficial trades, something other ranking systems by definition cannot do.

For example, in the eternal debate over who is better, Deron Williams or Chris Paul, we find that trading them would make both teams better off!

Here's the paper: http://ssrn.com/abstract=1935972

Comments as always much appreciated.


Top
 Profile  
 
PostPosted: Sat Oct 01, 2011 6:11 pm 
Offline

Joined: Sat Oct 01, 2011 5:55 pm
Posts: 2
Interesting paper.

You conclude:

Offensive ballhandling (preventing turnovers) has negative synergies with itself (-0.825) because
a lineup with one great ballhandler does not need another. Defensive ballhandling (creating
turnovers) has positive synergies with itself (0.307) because defenders who create turnovers feed
off each other and create more turnovers than they would individually
. Offensive scoring has
negative synergies with itself (-0.826) because players must share one ball. Similarly, defensive
scoring has negative synergies with itself (-0.284) because most defensive stands end with a stop
anyway
.


Is this to suggest that a team featuring Dwight Howard and Kevin Garnett would be worse than a team featuring Dwight Howard, all else equal? If yes, that philosophy would rock the basketball world. (Obviously, I'm using some extreme examples here, but you catch my drift, no?)


Top
 Profile  
 
PostPosted: Sat Oct 01, 2011 7:27 pm 
Offline

Joined: Sat Apr 16, 2011 7:40 am
Posts: 306
Location: Cambridge, MA
kelakos wrote:
Is this to suggest that a team featuring Dwight Howard and Kevin Garnett would be worse than a team featuring Dwight Howard, all else equal? If yes, that philosophy would rock the basketball world. (Obviously, I'm using some extreme examples here, but you catch my drift, no?)

It just means the sum of the impact is less than the individual talents, rather than more than the individual talents, which makes some sense

_________________
http://pointsperpossession.com/
@PPPBasketball


Top
 Profile  
 
PostPosted: Sat Oct 01, 2011 9:37 pm 
Offline

Joined: Thu Apr 14, 2011 11:10 pm
Posts: 4560
Hi Philip. Thanks for posting this article. It is quite interesting to see the approach and the 3 skill ratings for players and how they interact in lineups and how they could affect team moves.

I have a number of things I wanted to put out there to check to see if I am understanding what is being presented and to explore next steps. I'd appreciate your responses.

Is it true that total synergy of a lineup would be the sum of the overall net synergies of 10 player pairs and 21 skill synergies per pair for a total of 210 synergies?

Given the data on page 11, is it being asserted that every team in the league had the theoretical possibility of a 'best 5 player' lineup at least 7 points better than a lineup entirely composed of strictly no impact players and that the average is about 20 points?

What is the average lineup impact rating of the average lineup played in the league? Is it close to 12-15?

Using an appropriately weighted average opponent rating faced by the average starting lineup, how closely does average lineup impact rating of a starting lineup - average lineup impact rating of the average mix of opposing lineups compare to the actual average raw +/- posted by those lineups? Are your article's results for synergy, with no starting lineup showing more than a 10% change from sum of separate player ratings to combined lineup rating, assuming or asserting that any variance above that level in the actual +/- data is random noise due to sample size or is this an issue that needs further attention? I haven't calculated the average variance, but I believe it to be significant in size based on recent comparison of rank on separate RAPMs vs rank of that lineup's RAPM. See last post in this thread viewtopic.php?f=2&t=290&start=15)
210 synergies per lineup and no best 5 had a net synergy of more +1.2 or less than -0.8? That is a much tighter range than I would have expected or previously thought based on RAPM comparisons. It might be different for lesser lineups... is it or does the same modest range still apply?. Would it be fair to say your net synergies are asserted to be the true net synergy rather than observed and therefore do not need to lead to a match sum of separate player ratings and combined lineup ratings?

Have you systematically compared the overall values of SkillPM to RAPM or other APM or other boxscore stats? Would you be interested in presenting data on how well they do comparatively at predicting lineup performance? I would think that would be a very important step. If your SkillPM does best that would give it higher stature... for the purpose of predicting lineup performance.

The max positive PORP values are about the same height as traditional APM whereas RAPM's max values are only about half as high. Do you have can comment on the appropriateness of regression to the mean as practiced by RAPM but not traditional APM and I assume not by SkillPM either? To what extent does this choice interrelate with whatever average observed variance between expected performance based on separate player ratings (calculated as described above) and actual lineup +/- performance?

Would you be interested in comparing your SkillPMs comprehensively for each skill to RAPM's factors http://stats-for-the-nba.appspot.com/ , to the extent possible? How close are they on average? I checked the top 10s against each other. On offensive scoring there were 7 in both top 10s or real close. On the defensive scoring factor 5 were close to top 10 on each. 4 on offensive rebounding, 5 on defensive rebounding. 5 on own turnovers, just 3 on forced turnovers. I didn't check the size of the misses on all the other factors but for offensive scoring is was the ranking difference was about 45 spots. So some similarity, some differences in this limited look at top guys (where estimates might vary the most?). Some of the difference is probably due to the different length time periods and cutoffs to qualify. The match-up may be better than these numbers suggest.

Any chance you might share the full SPMs for the entire league to allow both full appreciation of your work but also aid further comparative work?

Do you have any opinion / view based on training & experience about the value of computing blended metrics for decision-making instead of going with a single metric? Say SkillPM and RAPM or a more involved blend with additional metrics? Is this potentially or likely an optimal strategy or mainly just risk-adverse (or do these coincide?)?

Have you looked at or are you interested in looking at average skill synergies for regular season vs playoffs and each compared with average skill synergies against elite teams in the regular season?

Do one-dimensional SkillPM players on average have a greater variance in actual impact on lineups based on whether they faced a negative skill synergy on their top skill in that particular lineup than well-rounded players?
Should opposing teams play then differently in these different circumstances for the one-dimensional player? Is there evidence that different strategies are producing different average results? Any difference in how well top one-dimensional SkillPM players and top well-rounded players do on average in the playoffs?

Why is Batum such a high and popular choice from the free agent pool when he is not listed as top 10 on overall SPM or any of the separate skill lists? Is he well-rounded? Is his particular skill profile just particularly well-matched with the skill deficiencies of many team's best 4s, given that in this exercise the best 4 (and 5) appear to be independent of position considerations? I would think a list with at least some position considerations would be more revealing and useful. What to conclude from Batum not making the Blazer top 5 on SPM and yet being a top "free agent" 5th man? Did he not qualify on time played or not on SPM? Same type discussion with Amir Johnson, Thaddeus Young and Hibbert. Only Hibbert had 1 top 10 skill showing, none made the top 10 SPM overall and yet are also among the most common optimizing choices to add to a team's top 4. Is it in part because they are tweeners and might best fill a gap left by a best 4 determined independent of position?

Have you seen David Sparks' work on player types here viewtopic.php?f=2&t=116 and at his site arbitrarian.wordpress.com/? Do you have any interesting in sorting players into types based on skills? Potentially, if you combined offensive and defensive rebounding into one skill set to reduce the complexity a bit and split skill sets simply into positive and negative performers I think that would yield 20 subgroups. Or one could group scoring and turnovers together on both sides of the court (and they do combine to represent initial offense before rebounding) and leave rebounding distinct on each side of the court and just have 12 subgroups. 20 might be more than most would want to handle. 12 seems pretty acceptable (to me).

Mutually beneficial trade search seems quite important- to get a sense of what is more likely to be mutually accepted. I have explored it using raw 4 four factor data and RAPM factors but without skill synergies. With synergies sounds like it might be modestly better. But if net skill synergy impact on lineups are pretty minor on average, it appears that simple skill level assessments of proposed lineups after trades would still be pretty good.

Have you looked at any of the last 5-10 biggest trades to draw a conclusion using your ratings and approach as to whether they were mutually beneficial and what percentage of big trades or trades in general are mutually beneficial and "who won" more?

Does the fact that teams tend to play other lineups besides their most used lineup as much or more minutes than the most used lineup (hopefully their best 5 but maybe not) make trading for a well-rounded player generally a better choice (providing what is missing from several key lineups, subbing out different lineups instead of just focusing optimizing one lineup an ddealin with one substitition) unless a team is extremely lacking in the skill that the one-dimensional player excels at and in one than one lineup?

Some of the "mutually beneficial trades" listed might surprise many people based on player reputation or boxscore stats but maybe less so if they looked at RAPM or your SPM values. Still I am somewhat intrigued by the BOS-MIN Rondo-Jefferson trade mentioned. Would that hypothetically have been for the 2010-11 season or 2009-10? Would you be willing to share their SPMs and the breakout like you did with Williams-Paul? Position considerations / team depth would seem to be important for the wisdom of this hypothetical or are you / SPM saying it would have been mutually beneficial even in full light of these factors? How beneficial was it- slight or significant and in whose favor?

What were the best mutually beneficial trades identified (if you are willing to say) and how big gain could both sides yield in the most beneficial such trades? Have any of these 222 happened yet, in any altered overall form? Logically this tool could be used for multiple player trades and other assets (draft picks & cash, if they were given average or team specific values for their situation / goals / stage in the competitive window).

Rich Cho is said to have a tool that gives every player a specific single number value for trade considerations. Your trade consideration approach would be quite different by emphasizing top lineup specific values in the new contexts (starting lineup or maybe extended to the basic rotation set of lineups). What were some of the worst recent big trades in terms of a team losing overall value (and to a lesser importance, hurting net synergy)?


Apart from what I mentioned above are there any other next steps you have identified for your research or additional questions that you've identified that you would be willing to mention? Thanks for consideration of these comments.


Top
 Profile  
 
PostPosted: Sun Oct 02, 2011 7:10 pm 
Offline

Joined: Fri Apr 15, 2011 12:37 am
Posts: 280
Philip - Is there any reason to think that SPM doesn't have the same inherent weaknesses that make APM a poor tool? Namely, terrible issues with collinearity that lead to poor predictive ability?


Top
 Profile  
 
PostPosted: Sun Oct 02, 2011 9:00 pm 
Offline

Joined: Thu Apr 14, 2011 11:24 pm
Posts: 323
I very much like the general conceptualization of the issues. A couple points of concern about the empirical results:

(1) What is the baseline for a team of "replacement players", specifically a team of such would produce a score differential of what? I ask for to calibrate your "PORP". Assuming that a team of replacement players is, as one would expect, well below average, this places the value of the elite players in a "non-ORP" framework well above that suggested by conventional APM, never mind RAPM.

(2) I do not understand the logical possibility of negative synergies. Perhaps I misunderstand the construct. You write: "Offensive ballhandling (preventing turnovers) has negative synergies with itself (-0.825) because a lineup with one great ballhandler does not need another." It is one thing to claim that acquiring a second, above-average ballhandler will exhibit decreasing returns. It is another to say that these are negative. Are you saying the latter? If so, I think there is a big problem, as it is theoretically incorrect. How can such a result obtain? Please, tell a story.


Top
 Profile  
 
PostPosted: Mon Oct 03, 2011 3:42 pm 
Offline

Joined: Mon Oct 03, 2011 3:33 pm
Posts: 14
Bobbofitos wrote:
kelakos wrote:
Is this to suggest that a team featuring Dwight Howard and Kevin Garnett would be worse than a team featuring Dwight Howard, all else equal? If yes, that philosophy would rock the basketball world. (Obviously, I'm using some extreme examples here, but you catch my drift, no?)

It just means the sum of the impact is less than the individual talents, rather than more than the individual talents, which makes some sense

That's exactly right. A team with Dwight Howard and Kevin Garnett is a lot better than a team with Dwight Howard.


Top
 Profile  
 
PostPosted: Mon Oct 03, 2011 3:44 pm 
Offline

Joined: Mon Oct 03, 2011 3:33 pm
Posts: 14
xkonk wrote:
Philip - Is there any reason to think that SPM doesn't have the same inherent weaknesses that make APM a poor tool? Namely, terrible issues with collinearity that lead to poor predictive ability?

SPM will have the same issue with collinearity. We addressed this issue in our paper by using four years of data instead of one.


Top
 Profile  
 
PostPosted: Mon Oct 03, 2011 3:54 pm 
Offline

Joined: Mon Oct 03, 2011 3:33 pm
Posts: 14
schtevie wrote:
(1) What is the baseline for a team of "replacement players", specifically a team of such would produce a score differential of what? I ask for to calibrate your "PORP". Assuming that a team of replacement players is, as one would expect, well below average, this places the value of the elite players in a "non-ORP" framework well above that suggested by conventional APM, never mind RAPM.


We rated the 360 players who played the most plays in our data set. We define "Replacement player" as all others. Replacement players include very good players such as Ty Lawson who were rookies and did not play enough to make the top 360. A team with replacement players would easily be the worst in the NBA.

schtevie wrote:
(2) I do not understand the logical possibility of negative synergies. Perhaps I misunderstand the construct. You write: "Offensive ballhandling (preventing turnovers) has negative synergies with itself (-0.825) because a lineup with one great ballhandler does not need another." It is one thing to claim that acquiring a second, above-average ballhandler will exhibit decreasing returns. It is another to say that these are negative. Are you saying the latter? If so, I think there is a big problem, as it is theoretically incorrect. How can such a result obtain? Please, tell a story.

It's semantics. We use the term negative synergies the same as decreasing returns. Here's a definition of "synergy": "The interaction or cooperation of two or more organizations, substances, or other agents to produce a combined effect greater than the sum of their separate effects." With negative synergies, the combined effect would be less than the sum of the separate effects.


Top
 Profile  
 
PostPosted: Mon Oct 03, 2011 5:16 pm 
Offline

Joined: Mon Oct 03, 2011 3:33 pm
Posts: 14
Crow wrote:
Is it true that total synergy of a lineup would be the sum of the overall net synergies of 10 player pairs and 21 skill synergies per pair for a total of 210 synergies?

No, the total synergy of a lineup is calculated as PORP(p1,p2,p3,p4,p5) - PORP(p1,rp,rp,rp,rp) - PORP(p2,rp,rp,rp,rp) - PORP(p3,rp,rp,rp,rp)- PORP(p4,rp,rp,rp,rp)- PORP(p5,rp,rp,rp,rp), where PORP is point differential vs. a team of replacement players, p1-p5 are players, and rp is a replacement player.

Crow wrote:
Given the data on page 11, is it being asserted that every team in the league had the theoretical possibility of a 'best 5 player' lineup at least 7 points better than a lineup entirely composed of strictly no impact players and that the average is about 20 points?

That's correct. More answers coming...


Top
 Profile  
 
PostPosted: Mon Oct 03, 2011 5:39 pm 
Offline

Joined: Mon Oct 03, 2011 3:33 pm
Posts: 14
Crow wrote:
Why is Batum such a high and popular choice from the free agent pool when he is not listed as top 10 on overall SPM or any of the separate skill lists? Is he well-rounded? Is his particular skill profile just particularly well-matched with the skill deficiencies of many team's best 4s, given that in this exercise the best 4 (and 5) appear to be independent of position considerations? I would think a list with at least some position considerations would be more revealing and useful. What to conclude from Batum not making the Blazer top 5 on SPM and yet being a top "free agent" 5th man? Did he not qualify on time played or not on SPM? Same type discussion with Amir Johnson, Thaddeus Young and Hibbert. Only Hibbert had 1 top 10 skill showing, none made the top 10 SPM overall and yet are also among the most common optimizing choices to add to a team's top 4. Is it in part because they are tweeners and might best fill a gap left by a best 4 determined independent of position?

Batum is #15 in overall SPM. His skill ratings are: Oballhandling 1.9 PORP, dballhandling 2.5 PORP, orebounding 1.3 PORP, drebounding -0.2PORP, oscoring 4.4 PORP and dscoring -2.0 PORP. The top 5 lists use the players who have participated in the most plays in our data set, not the players with the highest overall PORP. Same answer for Johnson, Young, and Hibbert. All are highly rated, but did not participate in enough plays to make the top 5 of our data set.


Top
 Profile  
 
PostPosted: Mon Oct 03, 2011 6:29 pm 
Offline

Joined: Thu Apr 14, 2011 11:35 pm
Posts: 151
Bobbofitos wrote:
kelakos wrote:
Is this to suggest that a team featuring Dwight Howard and Kevin Garnett would be worse than a team featuring Dwight Howard, all else equal? If yes, that philosophy would rock the basketball world. (Obviously, I'm using some extreme examples here, but you catch my drift, no?)

It just means the sum of the impact is less than the individual talents, rather than more than the individual talents, which makes some sense


Right. This could be the foundation for getting to the Holy Grail of being able to model how individual players' abilities interact to determine the team's outcomes. If you've got an average-ish team, adding Kareem Abdul-Jabbar helps you a lot. Adding Wilt Chamberlain helps you a lot. Adding BOTH Kareem and Wilt also helps you, but adding the second center doesn't help nearly as much as adding the first one did. If you've already got Kareem, you want to be adding Magic Johnson or Michael Jordan to your roster, not Wilt Chamberlain.


Top
 Profile  
 
PostPosted: Mon Oct 03, 2011 8:36 pm 
Offline

Joined: Fri Apr 15, 2011 12:02 am
Posts: 3816
Location: Asheville, NC
What if you have a 20%-winning team, and you add David Robinson and Tim Duncan?
The Spurs did that and became a 70% winning team.
Is there any way to predict these things?


Top
 Profile  
 
PostPosted: Mon Oct 03, 2011 8:42 pm 
Offline

Joined: Mon Oct 03, 2011 3:33 pm
Posts: 14
Crow wrote:
Have you systematically compared the overall values of SkillPM to RAPM or other APM or other boxscore stats? Would you be interested in presenting data on how well they do comparatively at predicting lineup performance?


We have not yet. The focus of our paper is to introduce a framework that evaluates synergies, not to discuss the pros and cons of yet another ratings metric. Our framework can certainly be used to predict lineup performance, but that topic is beyond the scope of the current paper.


Top
 Profile  
 
PostPosted: Mon Oct 03, 2011 8:47 pm 
Offline

Joined: Mon Oct 03, 2011 3:33 pm
Posts: 14
Crow wrote:
Do you have can comment on the appropriateness of regression to the mean as practiced by RAPM but not traditional APM and I assume not by SkillPM either? To what extent does this choice interrelate with whatever average observed variance between expected performance based on separate player ratings (calculated as described above) and actual lineup +/- performance?


I think that regression to the mean can be a legitimate technique to improve out-of-sample performance. In finance, we often use ridge techniques in a similar manner. For example, when forecasting the beta of a stock, we might "squeeze" our estimate towards 1. I think it's an empirical question, and since our paper is not focused on forecasting performance, we do not address this issue.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 35 posts ]  Go to page 1, 2, 3  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 7 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group