Hi Philip. Thanks for posting this article. It is quite interesting to see the approach and the 3 skill ratings for players and how they interact in lineups and how they could affect team moves.
I have a number of things I wanted to put out there to check to see if I am understanding what is being presented and to explore next steps. I'd appreciate your responses.
Is it true that total synergy of a lineup would be the sum of the overall net synergies of 10 player pairs and 21 skill synergies per pair for a total of 210 synergies?
Given the data on page 11, is it being asserted that every team in the league had the theoretical possibility of a 'best 5 player' lineup at least 7 points better than a lineup entirely composed of strictly no impact players and that the average is about 20 points?
What is the average lineup impact rating of the average lineup played in the league? Is it close to 12-15?
Using an appropriately weighted average opponent rating faced by the average starting lineup, how closely does average lineup impact rating of a starting lineup - average lineup impact rating of the average mix of opposing lineups compare to the actual average raw +/- posted by those lineups? Are your article's results for synergy, with no starting lineup showing more than a 10% change from sum of separate player ratings to combined lineup rating, assuming or asserting that any variance above that level in the actual +/- data is random noise due to sample size or is this an issue that needs further attention? I haven't calculated the average variance, but I believe it to be significant in size based on recent comparison of rank on separate RAPMs vs rank of that lineup's RAPM. See last post in this thread viewtopic.php?f=2&t=290&start=15
210 synergies per lineup and no best 5 had a net synergy of more +1.2 or less than -0.8? That is a much tighter range than I would have expected or previously thought based on RAPM comparisons. It might be different for lesser lineups... is it or does the same modest range still apply?. Would it be fair to say your net synergies are asserted to be the true
net synergy rather than observed and therefore do not need to lead to a match sum of separate player ratings and combined lineup ratings?
Have you systematically compared the overall values of SkillPM to RAPM or other APM or other boxscore stats? Would you be interested in presenting data on how well they do comparatively at predicting lineup performance? I would think that would be a very important step. If your SkillPM does best that would give it higher stature... for the purpose of predicting lineup performance.
The max positive PORP values are about the same height as traditional APM whereas RAPM's max values are only about half as high. Do you have can comment on the appropriateness of regression to the mean as practiced by RAPM but not traditional APM and I assume not by SkillPM either? To what extent does this choice interrelate with whatever average observed variance between expected performance based on separate player ratings (calculated as described above) and actual lineup +/- performance?
Would you be interested in comparing your SkillPMs comprehensively for each skill to RAPM's factors http://stats-for-the-nba.appspot.com/
, to the extent possible? How close are they on average? I checked the top 10s against each other. On offensive scoring there were 7 in both top 10s or real close. On the defensive scoring factor 5 were close to top 10 on each. 4 on offensive rebounding, 5 on defensive rebounding. 5 on own turnovers, just 3 on forced turnovers. I didn't check the size of the misses on all the other factors but for offensive scoring is was the ranking difference was about 45 spots. So some similarity, some differences in this limited look at top guys (where estimates might vary the most?). Some of the difference is probably due to the different length time periods and cutoffs to qualify. The match-up may be better than these numbers suggest.
Any chance you might share the full SPMs for the entire league to allow both full appreciation of your work but also aid further comparative work?
Do you have any opinion / view based on training & experience about the value of computing blended metrics for decision-making
instead of going with a single metric? Say SkillPM and RAPM or a more involved blend with additional metrics? Is this potentially or likely an optimal strategy or mainly just risk-adverse (or do these coincide?)?
Have you looked at or are you interested in looking at average skill synergies for regular season vs playoffs and each compared with average skill synergies against elite teams in the regular season?
Do one-dimensional SkillPM players on average have a greater variance in actual impact on lineups based on whether they faced a negative skill synergy on their top skill in that particular lineup than well-rounded players?
Should opposing teams play then differently in these different circumstances for the one-dimensional player? Is there evidence that different strategies are producing different average results? Any difference in how well top one-dimensional SkillPM players and top well-rounded players do on average in the playoffs?
Why is Batum such a high and popular choice from the free agent pool when he is not listed as top 10 on overall SPM or any of the separate skill lists? Is he well-rounded? Is his particular skill profile just particularly well-matched with the skill deficiencies of many team's best 4s, given that in this exercise the best 4 (and 5) appear to be independent of position considerations? I would think a list with at least some position considerations would be more revealing and useful. What to conclude from Batum not making the Blazer top 5 on SPM and yet being a top "free agent" 5th man? Did he not qualify on time played or not on SPM? Same type discussion with Amir Johnson, Thaddeus Young and Hibbert. Only Hibbert had 1 top 10 skill showing, none made the top 10 SPM overall and yet are also among the most common optimizing choices to add to a team's top 4. Is it in part because they are tweeners and might best fill a gap left by a best 4 determined independent of position?
Have you seen David Sparks' work on player types here viewtopic.php?f=2&t=116
and at his site arbitrarian.wordpress.com/? Do you have any interesting in sorting players into types based on skills? Potentially, if you combined offensive and defensive rebounding into one skill set to reduce the complexity a bit and split skill sets simply into positive and negative performers I think that would yield 20 subgroups. Or one could group scoring and turnovers together on both sides of the court (and they do combine to represent initial offense before rebounding) and leave rebounding distinct on each side of the court and just have 12 subgroups. 20 might be more than most would want to handle. 12 seems pretty acceptable (to me).
Mutually beneficial trade search seems quite important- to get a sense of what is more likely to be mutually accepted. I have explored it using raw 4 four factor data and RAPM factors but without skill synergies. With synergies sounds like it might be modestly better. But if
net skill synergy impact on lineups are pretty minor on average, it appears that simple skill level assessments of proposed lineups after trades would still be pretty good.
Have you looked at any of the last 5-10 biggest trades to draw a conclusion using your ratings and approach as to whether they were mutually beneficial and what percentage of big trades or trades in general are mutually beneficial and "who won" more?
Does the fact that teams tend to play other lineups besides their most used lineup as much or more minutes than the most used lineup (hopefully their best 5 but maybe not) make trading for a well-rounded player generally a better choice (providing what is missing from several key lineups, subbing out different lineups instead of just focusing optimizing one lineup an ddealin with one substitition) unless a team is extremely lacking in the skill that the one-dimensional player excels at and in one than one lineup?
Some of the "mutually beneficial trades" listed might surprise many people based on player reputation or boxscore stats but maybe less so if they looked at RAPM or your SPM values. Still I am somewhat intrigued by the BOS-MIN Rondo-Jefferson trade mentioned. Would that hypothetically have been for the 2010-11 season or 2009-10? Would you be willing to share their SPMs and the breakout like you did with Williams-Paul? Position considerations / team depth would seem to be important for the wisdom of this hypothetical or are you / SPM saying it would have been mutually beneficial even in full light of these factors? How beneficial was it- slight or significant and in whose favor?
What were the best mutually beneficial trades identified (if you are willing to say) and how big gain could both sides yield in the most beneficial such trades? Have any of these 222 happened yet, in any altered overall form? Logically this tool could be used for multiple player trades and other assets (draft picks & cash, if they were given average or team specific values for their situation / goals / stage in the competitive window).
Rich Cho is said to have a tool that gives every player a specific single number value for trade considerations. Your trade consideration approach would be quite different by emphasizing top lineup specific values in the new contexts (starting lineup or maybe extended to the basic rotation set of lineups). What were some of the worst recent big trades in terms of a team losing overall value (and to a lesser importance, hurting net synergy)?
Apart from what I mentioned above are there any other next steps you have identified for your research or additional questions that you've identified that you would be willing to mention? Thanks for consideration of these comments.