APBRmetrics

My apologies for carelessly having suggested that Dean said 25 points (as opposed to 25 wins) as the max effect of a player contribution. But still, such an effect is significantly greater than any in the xRAPM/RPM record. A notional increase of 25 wins implies substituting the contributions of a (36 mpg) "average" player (i.e. net 0 per 100 possessions) with someone providing mid 13s to 14s. And that guy hasn't yet shown up in the record.

hoopstudies wrote:George Karl was pretty significant, maybe not at 95%, but he did get a lot out of most of his players. I studied it when I was there with him. If you have a good coach, he will get more out of the players you bring in. That makes it easier to make trades. Were there players that he wouldn't maximize? Yes, and we figured out a bit of what kinds of players that would be. The not-so-smart ones. I didn't know that when I first got there.

So, Dean, what was your estimated effect, in terms of the (net?) points you attributed to Karl's coaching?

I don't think I ever estimated a total, but I'd guess in hindsight that he added 5-8 wins per season.

It was interesting making projections of players before we got them - either in the Draft or via other NBA teams - and seeing them exceed the projections.

"But i've seen things as high as 25 that's hard to discredit" in a discussion about how many wins a super star is worth.

fwiw definitely on the high side...

michael jordan's 87-88 season was the best by an SG over the past 4 decades in terms of wins generated on a per minute basis. that same year starting SGs like albert king (phi) and otis birdsong (njn) were quite bad in terms of wins generated on a per minute basis. when i simulate both for 40 min/g and 82 games on their respective teams, then replace them with jordan and repeat the sim, you see a 23-24 game improvement in W-L record over 82 games with both teams...

hoopstudies wrote:I don't think I ever estimated a total, but I'd guess in hindsight that he added 5-8 wins per season.

It was interesting making projections of players before we got them - either in the Draft or via other NBA teams - and seeing them exceed the projections.

Perhaps it's my bias owing to the Debacle of 2002, but it doesn't sound right to me that George Karl (or any other coach for that matter) would add five to eight wins per season, what corresponds, approximately, to improving team efficiency by 2 to 3 points per 100 possessions.

Being a suspicious person by nature, I have taken a closer look at the end of the Karl regime in Denver to see what (publicly available) xRAPM data suggest about the value he might have added.

So, the exercise was to look at the players not having started their careers in Denver that played in Karl's final season (2012-13) and who played more than 1000 minutes (an arbitrary cutoff). These were, in descending order of total minutes: Andre Iguodala, Danilo Gallinari, Andre Miller, Corey Brewer, Kosta Koufos, and JaVale McGee (who played in total about 63% of total mintues).

The two calculations then are to calculate the average change in player xRAPM (adding in the effect of Jeremias' RPM-based age adjustment) after the players' arrival to Denver and then the change in 2013-14 when Karl was no longer coach (and certain players were on other teams).

These two averages (the effect of Karl on acquired players and their development post-Karl) it seems to me should offer fair comment on his ability to add value as a coach.

So, just to be sure we are on the same page here, let me elaborate on the calculation. Consider a constituent player in the average, Andre Iguodala. His stint in Denver was but one year, the final year of Karl's tenure. In the previous year his xRAPM estimate was 5.3, then he dipped to +4.1 under Karl, then rose the following year in Golden State to +6.7. During this stage of his career, he was beginning to descend the aging curve, so eyeballing the graph posted in these here archives, I estimated that the age-adjustment was about -0.1 and -0.15 for the two years in question. And this would give AI's contribution to the Karl effect as -1.1 arriving and +2.75 departing.

So, what are the overall results?

The "Karl effect" on developing players (for "undrafted" players in his final year in Denver, then prorated over total team minutes) is calculated to be -0.01 points per 100 possessions. That is to say zero.

The "Karl effect" on his departure, by contrast, was +0.6 points per 100 possessions, what is to say that players did better by that amount than their aging curves would suggest not under his tutelage.

And the departing effect is in a real sense worse as it includes the effect of nasty injuries suffered by two of the players (Gallinari and McGee). Taking their contribution out and the average soars to +2.1.

So, I don't know how this approach based on public xRAPM data can be reconciled with the proprietary data that was the basis of Dean's estimate, but it certainly tells a different story.

Different methods, explained and not. You decide, or not.

Berri's method is fully laid out and available. You can replace his player metric with whatever you like and run the analysis. It ain't easy, but not much is easy in evaluating coaches.

Or you can just listen to Berri's conclusion that coaches don't matter at all. Then you probably disagree with everything in this article http://www.espn.com/espn/feature/story/ ... erie-board.

I would strongly disagree with any conclusion that says that coaches don't matter. If you have one coach who does things in an analytical way and another that doesn't - which would you think you'd want to work with in order to get wins?

George talked to me about 2002 only once and it was brief. I didn't want to ask about it. But you still have to remember to judge people based neither on their best nor worst times. He's a great coach, very smart, hard working, competitive. Cared about his staff and making his players better. Didn't care a lot about what people thought of him.

hoopstudies wrote:Berri's method is fully laid out and available. You can replace his player metric with whatever you like and run the analysis. It ain't easy, but not much is easy in evaluating coaches.

Dean, is there a link to Berri's coaching stuff you might have on hand? I don't recall it, from back in the day, as having been terribly interesting. There are two general approaches one can take. One is to estimate the direct contributions (what requires granular, "inside" information and what seems to be the basis of your faith in Karl) and the other is to treat the coaching contribution as a residual, having subtracted the contributions of players (the basis of my estimate of Karl's Denver swansong). Me, I have a bit more faith in the latter, generally (especially over many years of observation) as there is a great risk of spurious attribution of success/failure in the former.

hoopstudies wrote:Or you can just listen to Berri's conclusion that coaches don't matter at all. Then you probably disagree with everything in this article http://www.espn.com/espn/feature/story/ ... erie-board.

Funny you should have mentioned this article. Earlier today I had begun to slog through it, but just couldn't keep reading, only revisiting and finishing it upon your prompting. And as for me (my conclusion ain't based on anything Berri ever said) I don't disagree with the Kerr piece, rather it well illustrates the point that there is an incredible desire in the media and population at large to celebrate "Coaches and Coaching" but when you scratch just a bit below the surface, their actual effect, numerically, when proper context is offered, is so much smaller than the rhetoric implies.

hoopstudies wrote:I would strongly disagree with any conclusion that says that coaches don't matter. If you have one coach who does things in an analytical way and another that doesn't - which would you think you'd want to work with in order to get wins?

Since you brought up the instantaneous transformation of the GSW under Kerr, as related in this ESPN feature story, let's consider the facts, in terms of the framework I previously provided: looking at how well 2013-14 plus-minus estimates, plus aging effects, anticipated the strength of Kerr's first team in 2014-2015.

Using 2013-14 RPM for as many as players as possible (then filling the gap with xRAPM and rookie estimates of -3.0) adding aging estimates (from the graphs here: viewtopic.php?f=2&t=8308&hilit=aging) and utilizing actual 2014-15 minutes, what I get is and estimated GSW Offensive Efficiency of +2.1, a Defensive Efficiency of 5.7, for a Total of +7.8. This compares to an actual performance of +6.0, +4.2, and +10.2.

And from this, we infer a "Kerr Effect" of: +3.9 on Offense, -1.5 on Defense, and +2.4 overall.

Now, +2.4 is not bad. In fact, it is good! But what does it mean and what is the story behind it? Well, what we learn from the article is that his big innovation, elaborated over charcuterie, is that isolation basketball is inefficient! (By the way, when did this Eureka! moment come to our common awareness on this little forum?) But fairness is due, he surely is owed congratulations for improving his team's performance above baseline by +2.4 points per 100 possessions. Except...he's not. This is the fallacy of hagiography. This overall improvement of +2.4 isn't relative to a baseline of zero; it's relative to the contribution of Coach Mark Jackson. And this may be a zero baseline, but it may also be quite negative. And, it's a one-off.

But fairness is due only up to a point. One needs to take a cold, hard look at what potential "Coaching Contribution" really is. In this instance (as in a the sadly small number of instances where one can find "exceptional performances" in individual coaching seasons, where the gains are straightforwardly attributed to the painfully obvious opportunities afforded by increased three-point shooting) it is a one-off realization of on-court talent potential by eliminating its previous misutilization.

So, yes, coaching "matters", but it is best understood as a zero-sum enterprise, essentially independent from player contributions, and in a "perfectly competitive" world (where the drag of fraternal, coaching ideology and fanboy owners had zero effect and best practice would be rapidly adopted) the range of coaching contributions would be very small indeed.

And a final note about the risks of misattribution of coaching input: it is striking as a recent "historical" matter, how exceptional coaching has been attributed to historically successful teams and those only. This is a curious and counterintuitive empirical phenomenon; we all know that there have always been truly great players on mediocre and occasionally below-average teams. But not so with coaching, apparently... I encourage folks to go back into the googles and see the frequency with which Phil Jackson's name pops up as the coach uber alles. Does that conclusion ring true at all with the passage of Knick time?

Nope.

Coaches are considered "great" primarily because they are on very successful teams whose success is overwhelmingly determined by player talent endowments, and a bit of luck helps too. (And adding the Knicks to one's resume changes all that right quick.)

schtevie wrote:

hoopstudies wrote:Berri's method is fully laid out and available. You can replace his player metric with whatever you like and run the analysis. It ain't easy, but not much is easy in evaluating coaches.

Dean, is there a link to Berri's coaching stuff you might have on hand? I don't recall it, from back in the day, as having been terribly interesting. There are two general approaches one can take. One is to estimate the direct contributions (what requires granular, "inside" information and what seems to be the basis of your faith in Karl) and the other is to treat the coaching contribution as a residual, having subtracted the contributions of players (the basis of my estimate of Karl's Denver swansong). Me, I have a bit more faith in the latter, generally (especially over many years of observation) as there is a great risk of spurious attribution of success/failure in the former.

Well, that is exactly the approach that Berri takes. He projects the player value based on age (and a couple other factors), then looks at all the players who played with a coach and then not with that coach to see if the residuals are statistically consistent with a coach improving or worsening a team. A handful ended up significantly helping teams. George was not at 95% significant. A few others were, but George was positive.

The method is a good indirect way to evaluate coaches. It is somewhat dependent on the player value method, but probably not a ton.

I did not use the direct approach that you mention. I used the indirect, comparing a lot of players we acquired to projections I made before we acquired them, as well as players we traded away and how well they did then. I definitely tried to understand all that George did to communicate and cajole players into being better. I don't know what of those methods worked and didn't work - I wish I did. I just know guys usually exceeded my projections, which made it easier in the front office.

But if you hate George based on public perception, nothing I can say will change your mind. I've learned that.

Dean, I don't believe that there's anything I've said here (recently and going back deep into the mists of time) from which once could conclude that I "hate George based on public perception". My view here, then, now, and always, has been that coaching isn't that big of a factor in determining NBA outcomes, and George Karl fits very comfortably in that schema. So, if I say he ain't great, that ain't "hate", it's just "meh".

Any perceived bias on my part, however, might owe to my having been Boston-based for thirty years and having taken an understandable interest in the local club, where absurd ass-covering remarks at the expense of a local good guy (and excellent player) are understandably (?) viewed....dimly. But perhaps my recollections are incorrect, and Karl didn't try to foist responsibility for US performance in 2002 on specific players...

Whatever the case, that sad episode is actually a consistent and instructive point: perceptions of greatness in coaching are ever and always corollaries of team success, and incorrectly so. In 2002 (as my contemporaneous comments in the vaporized archives would corroborate, were they to exist) the world had substantively caught up to the US, to the point that the US couldn't just "mail it in" and expect to win. And whether that team was best selected for international basketball (surely not) was neither here nor there. Statistically, as I recall, they were still the best team there (y'know average point differential and all that) yet they finished...6th. So, one might infer that the "coaching" input, wasn't all that. That's not hate.

And as for "indirect" approaches for evaluating coaches, you mentioned here that Karl was good for five to eight wins at Denver, presumably on average. What is, again, about +3.0 points per 100 possession. I quasi-randomly picked - but did not cherry-pick - one convenient year (the last of his tenure, what is convenient for the obvious reasons) and found a plus-minus-based estimate that was, at best, zero. Admittedly, this is but one of his nine year tenure, and perhaps, according to this approach, averaging in the other 8 years' results would substantively raise the estimate. Maybe.

I provided Berri links. But when I said method not revealed I meant Dean's.

Dean: "I don't think I ever estimated a total..."

My method was simple. Make projection of a player's future and when we got them, did they get better than that projection? By how much?

That is effectively what Berri was doing, but he also looked at when players left the team, which is more thorough. I did look at that from time to time, but I never put together the systematic study.

rlee, not quite sure why you reposted this year-old link, but I am grateful for the reference and encourage one and all to (re?)listen. There are some implicit predictions (DeanO style, so by no means hard) that merit notice.