Can we tell if a team is playing well?

[ampersand5]

I recently wrote an article about tennis and randomness here that I think is also applicable to basketball - http://danfrank.ca/meditation-on-tennis/

My thesis is that many sporting outcomes (ie if a FT goes in, if a pass goes on target, possibly even if a team wins a game) are based on probabilistic randomness, and not someone playing above/below their natural skill level. My belief is that too often people judge if a player/team played well based on outcomes (winning or losing), when it is nearly impossible to distinguish between a player/team performing poorly/well, and a good/bad outcome based on probabilistic randomness.

So as an observer of an NBA game/playoff series, are there any signs we can look at to measure if a team is performing well/poorly, or should we attribute most results to randomness?

I imagine that many people on here think that playing poorly/well are measures that can be identified, so I would love to hear some examples or counter arguments.

[Mike G]

I played enough to recognize when I was in a funk, could hardly run without tripping, ball would bounce off my hands, etc.
Other days, I could do just about anything just by thinking about it.
Such days may be random, but the individual events are certainly not random.

[Crow]

Expected value of shots says a lot about quality of maneuver and shot judgment, own and opponent. If the groundwork were done on expected risk for all passes one might be able to say similar for own and opponent turnovers. Same for rebounding for rebounds where you and opponent were positioned for a rebounding opportunity. Fouls for and against and differential are harder to call more earned or random.

[Nathan]

I've thought a little bit about this. If a team starts out the season, say, 10-0, can we look at box score stats to get insight into whether or not the team's "for real"?

If the dominant factor in their wins was that their opponents shot a combined 14% from 3, it seems reasonable to think it might be a fluke. In contrast if they've been winning by playing solid all-around basketball (e.g. outperforming their opponents in most/all statistical categories by a modest margin), it seems like they might be a legitimately great team.

Last year, the Warriors allowed a league-best 31.4% opponent 3 point shooting before the all-star break. After the all-star break, they allowed 36.0% 3 point shooting, around the league average. That's a significant difference, more than 3 points per game. Could we have anticipated this?

Has anyone looked into this kind of thing? In particular, most team rating systems look at w/l record and/or point differential, along with a couple other factors, but I don't know of any that incorporate box score stats in any way. Could box score stats have some predictive power for a team's future performance?

[Nate]

So as an observer of an NBA game/playoff series, are there any signs we can look at to measure if a team is performing well/poorly, or should we attribute most results to randomness?

If you're talking about something formal, like statistics, then the usual approach is to make some educated guesses about the nature of the randomness, and then use some techniques to try to subtract or cancel that noise from the outcomes.

For example, Crow's argument is based on the idea that player positioning is somehow less noisy than shot outcome, so if we can measure and evaluate player position instead of traditional box scores, then we might get a less noisy evaluation of team (or player) performance.

A more typical and general approach is to assume that the world is roughly a large number independent and identically distributed events so that outcomes are normally distributed, and then do some kind of inference based on those assumptions.

Such days may be random, but the individual events are certainly not random.

Everything that I've seen suggests that, in professional sports, momentum and clutch factor are largely narrative fallacies.

[Crow]

Thinking about it a bit more, one could compare ftas received / given up to "expected" based on average ft rate for the sum of the values for the mix of play types run, like for the other factors. They get to quality of strategy (and movement). The prep before the "execution". Execution is a big part of quality of play. The quality of your prep is some indication of quality of play but the remainder is very important and still about quality. Quality and randomness, hard to separate. If the quality of the strategic prep is far higher or lower than the results it might be that randomness associated with the execution played a decisive role in the game, but it could also be bad execution. Probably would be good to see the average quality of strategic prep and the volatility of it (sometimes might do the right thing more often than average via randomness instead of conscious quality of effort) and the same for all the execution variables and check out the coincidences (and the lineups and the opponent, etc.)

Use the microdata from player tracking. Did the jumpers not fail because of "randomness" or dis players execute the release angle and motion different / worse than average? Were the passes & dribbles lost & stolen as tight and safe and timely as average or does the data say they were wide, high and / or too late or early? You might have been in average rebound position and lost more rebounds than usual fir what looks like random chance but what does the data say about your footwork, jump, leverage, physicality, hands on those plays? On fouls earned and given, what is the data on degree of contact created? Randomness is something to acknowledge but the more you study and understand and the more you strive for perfection the less it should be used to end the examination and effort. If you knew the perfect serve mechanics and executed perfectly there would be no randomness left (for the serve, not the return). One could say "randomness" is imperfection of strategy & execution and not a constant, independent or inevitable force (strictly speaking). Typical, varying influence to be fought and accepted with humility in defeat after the fact by flawed actors but not before.

I accept randomness at the level of random interactions far more easily than randomness as something different than quality of individual effort. Randomness in individual effort exists because nobody or almost nobody know what perfect or truly optimal is even at the mechanical level, much less the interactive chess level. But some are ahead and there is room for more edge finding. How close are the release angle teachings of various shot gurus (presumably based on experience and subjective judgment) and the recommendation produced by statistical queries on hundreds of thousands or millions of shots? The impact of analytics is probably in its early stages, if you assume players and coaches will listen & apply (and ask questions too).

Caveat: I guess there is some randomness in the ball pressure, atmospheric temperature, humidity, micro wind conditions, elevation, etc. that could affect individual performance. I know the NBA and USTA have standards for balls and do some testing & quality control. But have either ever tested the air dynamics on courts and done anything to reduce the variations? I assume the answer is little, none or at least no proven reduction in volatility in court air dynamics. A devious court owner- with results from a research study- could try to alter air dynamics on one or both sides of court selectively during a match with side changes disadvantage or advantage shot performance thru adjustment in the levels of fans / air circulation vents around the court.

[Mike G]

Such days may be random, but the individual events are certainly not random.

Everything that I've seen suggests that, in professional sports, momentum and clutch factor are largely narrative fallacies.

I wouldn't refer to it as "momentum" or lack thereof; rather, is the player or team "playing well", as in the thread title.
"Clutch" is yet another tangential concept. A good player or team may be concluding a bad game, yet have the confidence and ability to finish strongly.

Even in the most consistently measurable performances -- track and field, golf? -- an athlete has good and bad days, weeks, months, and years.
You could keep saying "He's really better than this, and he's sure to get back to form" ... until such time as you say, "Well, he had a bad year (or month, etc)".

[ampersand5]

I played enough to recognize when I was in a funk, could hardly run without tripping, ball would bounce off my hands, etc.
Other days, I could do just about anything just by thinking about it.
Such days may be random, but the individual events are certainly not random.

While I'm skeptical of the claim that some days you could do just about anything by thinking about it, that's only relevant for first person experiences where you have additional information (your brain) to make an assessment on performance. Team performance is still uncertain.

Additionally, even if we accept the claim that you have those amazing/awful days, that doesn't negate the randomness of individual events for all other "ordinary" days.

[ampersand5]

I've thought a little bit about this. If a team starts out the season, say, 10-0, can we look at box score stats to get insight into whether or not the team's "for real"?

If the dominant factor in their wins was that their opponents shot a combined 14% from 3, it seems reasonable to think it might be a fluke. In contrast if they've been winning by playing solid all-around basketball (e.g. outperforming their opponents in most/all statistical categories by a modest margin), it seems like they might be a legitimately great team.

Last year, the Warriors allowed a league-best 31.4% opponent 3 point shooting before the all-star break. After the all-star break, they allowed 36.0% 3 point shooting, around the league average. That's a significant difference, more than 3 points per game. Could we have anticipated this?

Has anyone looked into this kind of thing? In particular, most team rating systems look at w/l record and/or point differential, along with a couple other factors, but I don't know of any that incorporate box score stats in any way. Could box score stats have some predictive power for a team's future performance?

Tom Tango did analysis on randomness and W-L records for each sport, which can be found here: http://www.insidethebook.com/ee/index.p ... s_leagues/
here is some additional Phil Birnbaum commentary: http://blog.philbirnbaum.com/2013/01/lu ... dings.html

You posed a fascinating question about GSW, and I'm curious to learn what other people think about it.

[ampersand5]

Thinking about it a bit more, one could compare ftas received / given up to "expected" based on average ft rate for the sum of the values for the mix of play types run, like for the other factors. They get to quality of strategy (and movement). The prep before the "execution". Execution is a big part of quality of play. The quality of your prep is some indication of quality of play but the remainder is very important and still about quality. Quality and randomness, hard to separate. If the quality of the strategic prep is far higher or lower than the results it might be that randomness associated with the execution played a decisive role in the game, but it could also be bad execution. Probably would be good to see the average quality of strategic prep and the volatility of it (sometimes might do the right thing more often than average via randomness instead of conscious quality of effort) and the same for all the execution variables and check out the coincidences (and the lineups and the opponent, etc.)

Use the microdata from player tracking. Did the jumpers not fail because of "randomness" or dis players execute the release angle and motion different / worse than average? Were the passes & dribbles lost & stolen as tight and safe and timely as average or does the data say they were wide, high and / or too late or early? You might have been in average rebound position and lost more rebounds than usual fir what looks like random chance but what does the data say about your footwork, jump, leverage, physicality, hands on those plays? On fouls earned and given, what is the data on degree of contact created? Randomness is something to acknowledge but the more you study and understand and the more you strive for perfection the less it should be used to end the examination and effort. If you knew the perfect serve mechanics and executed perfectly there would be no randomness left (for the serve, not the return). One could say "randomness" is imperfection of strategy & execution and not a constant, independent or inevitable force (strictly speaking). Typical, varying influence to be fought and accepted with humility in defeat after the fact by flawed actors but not before.

I accept randomness at the level of random interactions far more easily than randomness as something different than quality of individual effort. Randomness in individual effort exists because nobody or almost nobody know what perfect or truly optimal is even at the mechanical level, much less the interactive chess level. But some are ahead and there is room for more edge finding. How close are the release angle teachings of various shot gurus (presumably based on experience and subjective judgment) and the recommendation produced by statistical queries on hundreds of thousands or millions of shots? The impact of analytics is probably in its early stages, if you assume players and coaches will listen & apply (and ask questions too).

Caveat: I guess there is some randomness in the ball pressure, atmospheric temperature, humidity, micro wind conditions, elevation, etc. that could affect individual performance. I know the NBA and USTA have standards for balls and do some testing & quality control. But have either ever tested the air dynamics on courts and done anything to reduce the variations? I assume the answer is little, none or at least no proven reduction in volatility in court air dynamics. A devious court owner- with results from a research study- could try to alter air dynamics on one or both sides of court selectively during a match with side changes disadvantage or advantage shot performance thru adjustment in the levels of fans / air circulation vents around the court.

A lot of good points here. I will respond later when I have some more time, thanks.

[Nate]

...
You posed a fascinating question about GSW, and I'm curious to learn what other people think about it.

We can start with the null assumption that 3-point attempts are IID and have a natural rate of 1/3. (The total average against GSW is very close to that.)

Then the variance of shots made per shot attempted is 1/3 * (2/3)^2 + 2/3 * (1/3)^2 = 6/27=2/9.

Now, with 54 games before the all-star break, and roughly 25 FGA per game on average we expect around 1350 3pt FGA, and a standard deviation around 17.3 shots made. The 1/3 hypothesis gives a prediction of 450 made, the actual rate says 424 - so 1.5 standard deviations off. A result at least that extreme has roughly 1/15 chance of happening under the null hypothesis

With 28 games after the all-star break, we expect 233 shots made on 700 attempts with a standard deviation of 12.5 shots made and the actual rate is 252. Once again about 1.5 standard deviations off. So again, roughly 1/15.

So under the 'just luck' hypothesis, it's a 1 in 225 long shot. Not impossible, but certainly to the point where you'd think something did change around that time.

[Kevin Pelton]

Has anyone looked into this kind of thing? In particular, most team rating systems look at w/l record and/or point differential, along with a couple other factors, but I don't know of any that incorporate box score stats in any way. Could box score stats have some predictive power for a team's future performance?

Yeah, I put together a shooting-neutral defensive rating that uses league average for opponent 3-point percentage and free throw percentage. However, when I looked at it at some point midseason, it didn't improve predictive power for final performance. That might have been a fluke of the specific season/point I used, and maybe I need to regress opponent shooting somewhat to the mean instead of just using league average, but it didn't work as I expected.

[Nathan]

Has anyone looked into this kind of thing? In particular, most team rating systems look at w/l record and/or point differential, along with a couple other factors, but I don't know of any that incorporate box score stats in any way. Could box score stats have some predictive power for a team's future performance?

Yeah, I put together a shooting-neutral defensive rating that uses league average for opponent 3-point percentage and free throw percentage. However, when I looked at it at some point midseason, it didn't improve predictive power for final performance. That might have been a fluke of the specific season/point I used, and maybe I need to regress opponent shooting somewhat to the mean instead of just using league average, but it didn't work as I expected.

Thanks for the response! I wonder if it's a noise/small sample size issue, or if there's really nothing substantial to be gained through this approach.

If someone wanted to turn this into a project, they could try running a linear model where the "x" values are team and opponent box score stats and scoring margin through the first 41 games of a season, and "y" values are scoring margin in the last 41 games of the season. With data from 2000-01 on, or so, there would be ~500 data points, probably enough to determine if any box score stats have significant predictive power.

EDIT: I would also be happy to do this myself if someone suggests an easy way to get this data, that is, team and opponent stats for the first and last 41 games of a season.

[Crow]

If all you want is first half stats / net pt. margin vs second half of season data, this site might do http://www.dougstats.com/
Select season and team. Shows full season but click on "last 10 games". After season is over that second set of data is really last 41 games. Subtract from full season numbers to get first half of season data.

[sndesai1]

or just go to http://www.basketball-reference.com/tea ... 6/gamelog/ for each team