Two articles from Ian Levy:
http://www.hickory-high.com/?p=2006
http://www.hickory-high.com/?p=2080
Useful data that raises various questions.
Coaches and lineup optimization
-
Bobbofitos
- Posts: 306
- Joined: Sat Apr 16, 2011 7:40 am
- Location: Cambridge, MA
- Contact:
Re: Coaches and lineup optimization
How is this useful & what questions does it raise?Crow wrote:
Useful data that raises various questions.
http://pointsperpossession.com/
@PPPBasketball
@PPPBasketball
Re: Coaches and lineup optimization
Rob, did you read my comments there? That gives a flavor of why I think it is useful.
Have you looked at the data? Do you want to?
I hope you might forgive me for making "the mistake" of being brief here (late on a Friday night) and thinking the article link and the comments there would be an ok start to allow others to find & consider the data and value for themselves.
Stating the obvious, you can see which coaches had higher and lower correlations between minutes of lineup usage and performance. You can see the % of lineups and % of minutes that were positive. For career and you can see who had consistent performance on these measures year to year and who swung dramatically. It is not all exactly as I would have expected based on reputations or other coaching data.
Then you can dig into the year to year roster changes and lineup usage changes and lineup performance changes to begin to try to piece together what happened, how much was related to lineup coaching. I briefly started down that road with the Hawks, Spurs and Thunder.
It raises questions about going further to look at regular season vs playoffs and coaching resulting with top 5 lineups vs the rest. And perhaps more, with more review.
Have you looked at the data? Do you want to?
I hope you might forgive me for making "the mistake" of being brief here (late on a Friday night) and thinking the article link and the comments there would be an ok start to allow others to find & consider the data and value for themselves.
Stating the obvious, you can see which coaches had higher and lower correlations between minutes of lineup usage and performance. You can see the % of lineups and % of minutes that were positive. For career and you can see who had consistent performance on these measures year to year and who swung dramatically. It is not all exactly as I would have expected based on reputations or other coaching data.
Then you can dig into the year to year roster changes and lineup usage changes and lineup performance changes to begin to try to piece together what happened, how much was related to lineup coaching. I briefly started down that road with the Hawks, Spurs and Thunder.
It raises questions about going further to look at regular season vs playoffs and coaching resulting with top 5 lineups vs the rest. And perhaps more, with more review.
Re: Coaches and lineup optimization
Of the top 12 coaches on biggest positive difference between % of lineup minutes that were positive - % of lineups that were positive (on average) (discussed in article 2), 11 were coaches of the top 14 defenses last season. Those 12 coaches were in the top 14 defense 5 times more frequently than the rest of coaches. I think that is useful information.
7 of these had career win %s of .600+. Only 2 didn't. The top 12 coaches on this difference were 5-6 times more likely to have a .600+ career record. It might be worthwhile spending for a proven winner.
Of the 14 coaches with a correlation above .500 between +/- performance and lineup minutes assigned over a 4 year period (discussed in article 1) at least 10 (maybe 12) were above average defensive coaches. It might be worthwhile to favor a strong defensive coach.
If Ian proceeds to produce playoff data and data for top 5 lineups and the rest of lineups, it will be possible to see if defensive coaches hold most of the top performances there as well or not.
7 of these had career win %s of .600+. Only 2 didn't. The top 12 coaches on this difference were 5-6 times more likely to have a .600+ career record. It might be worthwhile spending for a proven winner.
Of the 14 coaches with a correlation above .500 between +/- performance and lineup minutes assigned over a 4 year period (discussed in article 1) at least 10 (maybe 12) were above average defensive coaches. It might be worthwhile to favor a strong defensive coach.
If Ian proceeds to produce playoff data and data for top 5 lineups and the rest of lineups, it will be possible to see if defensive coaches hold most of the top performances there as well or not.
Re: Coaches and lineup optimization
Of the top 12 coaches on biggest positive difference between % of lineup minutes that were positive - % of lineups that were positive (on average) only 7 were coaches of the top 14 offenses last season. Offense is important too but top performance on this measure was accompanied by top performance on defense 60% more often. Top offensive performance might come in bigger bursts from a subset of lineups and minutes. Detail that would probably be useful to check into further.
For the top 50 lineups on minutes in the league, the correlations of offensive and defensive efficiency and +/- performance were nearly identical at +.71. So while it seems that defense delivers its beneficial impact more broadly across minutes than offense (probably not a surprise that defense is more team / system while offense is more dependent on top players being on the floor in the right combinations), both are obviously very important and equally capable of affecting the net performance. It might help (at least on average) to optimize lineup set construction by approaching it with a strategy of maximizing minutes for strong defensive lineups that are edge producing and focusing on identifying specific top offensive efficiency units (and contexts) that are max edge producing. If you can increase the minutes of the top offensive efficiency units without losing as much on total edge produced by the strong defensive units, you would want to do that further adjustment. The underlying data is mostly real small sample but I'd think a team would want to explicitly consider dozens even gundreds of variations on lineup sets (periodically) instead of locking into one approach long-term but may not have optimal to begin with or may not stay that way for long. There could be frictional costs from monkeying around with rotations but to me using the regular season to gather more information to try to optimize your lineup set for the playoffs better than others teams seems pretty important. I guess teams have to weigh the cost & benefits of such research against the cost & benefits of playoff seed. There might be an optimal / compromise level of lineup research. Obviously you'd be guessing a bit about how long to research each lineup and how far to bump it up or when to leave it stable or eliminate it. Do the best coaches / teams experiment / adjust lineup levels more or less than less successful coaches? Break the data in months and you could probably find out more.
For the top 50 lineups on minutes in the league, the correlations of offensive and defensive efficiency and +/- performance were nearly identical at +.71. So while it seems that defense delivers its beneficial impact more broadly across minutes than offense (probably not a surprise that defense is more team / system while offense is more dependent on top players being on the floor in the right combinations), both are obviously very important and equally capable of affecting the net performance. It might help (at least on average) to optimize lineup set construction by approaching it with a strategy of maximizing minutes for strong defensive lineups that are edge producing and focusing on identifying specific top offensive efficiency units (and contexts) that are max edge producing. If you can increase the minutes of the top offensive efficiency units without losing as much on total edge produced by the strong defensive units, you would want to do that further adjustment. The underlying data is mostly real small sample but I'd think a team would want to explicitly consider dozens even gundreds of variations on lineup sets (periodically) instead of locking into one approach long-term but may not have optimal to begin with or may not stay that way for long. There could be frictional costs from monkeying around with rotations but to me using the regular season to gather more information to try to optimize your lineup set for the playoffs better than others teams seems pretty important. I guess teams have to weigh the cost & benefits of such research against the cost & benefits of playoff seed. There might be an optimal / compromise level of lineup research. Obviously you'd be guessing a bit about how long to research each lineup and how far to bump it up or when to leave it stable or eliminate it. Do the best coaches / teams experiment / adjust lineup levels more or less than less successful coaches? Break the data in months and you could probably find out more.
Re: Coaches and lineup optimization
Of the Mav's 6 biggest minute positive lineups on raw +/- last season, the 3 with the lowest sum of player average usage had the best offensive efficiencies and the 3 with the highest sum of player average usage had the lowest offensive efficiencies. Maybe random or maybe in the lineups with lower sum of player average usage the players knew the size of their role and had the opportunity to play that way while in the lineups with higher sum of player average usage (above a sum of 104 and an average of almost 110) maybe there was too much competition for shots or uncertainty who has going to get them? Obviously this is speculative and is just one team. Review of the tape might help, if done carefully.
The 3 lower usage sum lineups also had 3 of the 4 best defensive efficiencies. It seems like an art to assemble a strong lineup on offense with the right usage mix and on defense at the same time. Those who suggest or imply that lineups don't matter much and that players are "who they are" and that lineup performance is mostly or entirely noise seem to me to be oversimplifying things. I am still not sure exactly how important lineups are generally and how reliably consistent they are or can be in certain cases, but I want to know more and think it is probably worth further research.
Dallas' 6 best big minute lineups averaged 3.7 of their biggest minute players. 3 were clearcut best players by position using player APM. Kidd and Marion were not, yet the lineups with them worked well. Did they performed their role well in those lineups and not others or were they a drag? More research of player stats while in a specific lineup are needed. No one to my knowledge has yet made players stats while in a specific lineup available broadly and publicly. That would be a useful next step if any set up to do so, would.
How much will Barea, Beaubois or Fernandez play PG instead of Kidd? How much with Butler or Brewer play instead of Marion? They have options and seem to tend to pick fairly good ones. Pick a little less well (and that might happen with more options) and / or don't get as "hot" (at player level, or coaching lineup matchup choice) and / or encounter teams with better lineup management & hotness and ultimate victory may go to some other team who picked lineups better and / or got hot.
The 3 lower usage sum lineups also had 3 of the 4 best defensive efficiencies. It seems like an art to assemble a strong lineup on offense with the right usage mix and on defense at the same time. Those who suggest or imply that lineups don't matter much and that players are "who they are" and that lineup performance is mostly or entirely noise seem to me to be oversimplifying things. I am still not sure exactly how important lineups are generally and how reliably consistent they are or can be in certain cases, but I want to know more and think it is probably worth further research.
Dallas' 6 best big minute lineups averaged 3.7 of their biggest minute players. 3 were clearcut best players by position using player APM. Kidd and Marion were not, yet the lineups with them worked well. Did they performed their role well in those lineups and not others or were they a drag? More research of player stats while in a specific lineup are needed. No one to my knowledge has yet made players stats while in a specific lineup available broadly and publicly. That would be a useful next step if any set up to do so, would.
How much will Barea, Beaubois or Fernandez play PG instead of Kidd? How much with Butler or Brewer play instead of Marion? They have options and seem to tend to pick fairly good ones. Pick a little less well (and that might happen with more options) and / or don't get as "hot" (at player level, or coaching lineup matchup choice) and / or encounter teams with better lineup management & hotness and ultimate victory may go to some other team who picked lineups better and / or got hot.
Re: Coaches and lineup optimization
Finally finished running the numbers for the playoffs. There was a ton of variability from season to season, even more so than what the regular season shows. Surprisingly Mike Brown in 2009 came up as the top performance for an individual season.
http://www.hickory-high.com/?p=2436
If anyone has thoughts on continuing this line of work I'd love to hear them. Controlling for things like injuries, roster changes and eventually, matchups, seem like the next steps, but may be beyond my meager analytical skills.
http://www.hickory-high.com/?p=2436
If anyone has thoughts on continuing this line of work I'd love to hear them. Controlling for things like injuries, roster changes and eventually, matchups, seem like the next steps, but may be beyond my meager analytical skills.
Re: Coaches and lineup optimization
Also gave Crow a shout-out towards the end of the post. I hope I didn't take away any of his mystique.
Re: Coaches and lineup optimization
Maybe you can compare it to Jerry's adjusted coaching +/-.
Re: Coaches and lineup optimization
Thanks for the playoff data Ian and the shout-out.
Evan, I also thought about comparing Ian's data with Jerry's RAPM.
I guess now that I hadn't done it.
So I compared the correlation of lineup net rating to minutes given in Ian's first article for 45 recent coaches in the last 4 years to Jerry's RAPM for coaches for the last 4-5 years (saved from spring 2011) and the correlation was about .46. The correlation with offensive RAPM was slightly negative. The entire (though modest) correlation was essentially with defensive RAPM (there was a slight rounding issue for the rest).
Evan, I also thought about comparing Ian's data with Jerry's RAPM.
I guess now that I hadn't done it.
So I compared the correlation of lineup net rating to minutes given in Ian's first article for 45 recent coaches in the last 4 years to Jerry's RAPM for coaches for the last 4-5 years (saved from spring 2011) and the correlation was about .46. The correlation with offensive RAPM was slightly negative. The entire (though modest) correlation was essentially with defensive RAPM (there was a slight rounding issue for the rest).