Berri Changes Value of Defensive Rebounds in WP

[mtamada]

Anybody have thoughts about this study?

That article instantly got on my bad side by focusing on player rankings, and even worse calculating the standard deviation of the ranks. Bad statistical methodology. A better way of looking at how similar or dissimilar the player rating systems are to each other is to look at correlations, or possibly normalized measures (z-scores). Rankings are bad because the #2 and #3 ranked players could be really close to each other (maybe Kobe averages 27.9 points per game and Lebron averages 27.8 points), or far away (maybe the numbers are 27.9 and 24.9). Either way, it's much more informative and for almost all purposes more accurate to use the actual numbers (27.9, 27.8, 24.9, etc.) instead of the cruder and less informative ranks (#1, #2, #3).

When we're evaluating players, then except for specialized purposes such as handing out all-NBA awards and the like, ranks are not what we should focus on, instead we should focus on measures of player quality/ability. Ie. look at Win Shares, PER, WP48, etc. instead of the rankings that those measures produce.

If one truly does have to analyze data which are ordinal in nature, such as players' ranks, there are specialized order statistics that should be used, not the standard deviation.

Thumbs down on the article, I didn't bother to look at the results or conclusions in detail because I wouldn't believe them anyway. He evidently had nice data on WinShares, etc. and instead of comparing them in an interesting way, he frittered them away by doing the silly rankings thing.

[bbstats]

I wouldn't say *bad* methodology. Just not ideal. Z-scores don't end up being tons different except that they weed out large disparity in players near the mean (since there are so many, ranking deviation is common).

[Crow]

Thanks for the comments highlighting the use of rankings over ratings and that impact of that.

[Bobbofitos]

Anybody have thoughts about this study?
http://weaksideawareness.wordpress.com/ ... /#comments
I haven't stared at it enough and am not really in the mood to do so right now but thought I'd ask if anybody had thoughts to make decoding quicker or make the task more interesting.

Kinda funny that Yi Jianlian was consistently terrible in a bunch of seasons. And yet he keeps getting NBA money...

[mtamada]

I wouldn't say *bad* methodology. Just not ideal. Z-scores don't end up being tons different except that they weed out large disparity in players near the mean (since there are so many, ranking deviation is common).

Correct, which is one of the main reasons why it's such a mistake to focus on rankings. There are a lot more players near the mean than at the extremes (or than at the top extreme at least). Moreover their deviations (as measured by their rank under various systems) will be large because there are so many players that are close to each other.

LeBron's rank isn't going to vary by more than, what, maybe 10 ranking places? (I.e. just about any sensible evaluation system is going to rank him as a Top 10 player, indeed I would say that any system should rank him as Top 5.) So the top players are not going to have large deviations.

The average players on the other hand might have huge deviations in their ranks, a player might be #150 under WinShares and #200 under PER. Looks like the two systems disagree hugely with each other? Probably not, this player might have a z-score of 0.1 under one system and -0.1 under the other -- i.e. they basically agree that he's an average player -- but due to the inherent inaccuracy of using rankings, the systems will appear to be in huge disagreement.

And there are a lot more Joe Average players than LeBrons in the league. This comparison of different player evaluation systems is biased towards finding big differences when the differences in fact are small. (And conversely may fail to detect big differences amongst the rankings of the best players, whose ranking deviations will tend to be small, as in the LeBron example.)

[xkonk]

The idea is similar to something posted at sport skeptic, toward the bottom of the article http://sportskeptic.wordpress.com/2011/ ... get-along/.

[Mike G]

At basketball-reference, we can see playoff summaries:
http://www.basketball-reference.com/pla ... _2012.html

In the miscellaneous team statistics, both the Jazz and the Knicks are seen to have 0.1 PW (and 1.9 PL) after 2 games.
But on their team pages, both teams' players WS sum to -0.5 for these 2 games.
http://www.basketball-reference.com/teams/NYK/2012.html

Aren't player Win Shares supposed to sum to team Pythagorean-expected wins?
The Heat and Spurs rosters both total 2.5 WS after 2 games.

[Crow]

Rounding to tenths and doing it all a certain way can produce aberrations, especially right at the start of the counting.

[Mike G]

Rounding to tenths wouldn't raise or lower a team's pyth wins by 0.6, unless 12 players all rounded up or down by the max (.05).
In the case of the 4 teams named, the odds would be astronomical.
Teams in non-blowout series don't exhibit this tendency.

Apparently, while PWins are constrained between zero and 1 per game, Win Shares are not.
In the course of a full season, or much more than a couple of games, WS will not likely go beyond 0% or 100% of games played.

Then again, this year's Bobcat's are shown with 7 pyth wins.
Of their players, 10 have a total 6.9 WS; another 6 players total -3.5
I guess I don't know what this means.

[bbstats]

@Mike - Read the notes at the bottom http://www.basketball-reference.com/about/ws.html

It used to force fit to actual wins, now it just estimates I guess.

[Mike G]

I know they used to force fit to actual wins, and I'd thought that now they estimated pythagorean wins.
Differing by some .3 (wins) per game, between WS and PyW, does not seem to be a very good estimate.

[mtamada]

The idea is similar to something posted at sport skeptic, toward the bottom of the article http://sportskeptic.wordpress.com/2011/ ... get-along/.

Yes, that was a much better analysis, using correlations instead of rankings. OTOH some of that analysis had numerous flaws, which we discussed in the thread about using a blend to retrodict, although I don't know if those flaws apply to this analysis.

[xkonk]

There was no blending in that article, so the only concerns would be about the data set itself. That's public, so the analysis can be replicated or expanded by anyone who wants to.

[wilq]

Anybody have thoughts about this study?
http://weaksideawareness.wordpress.com/ ... /#comments

I wouldn't call it a study... those are just facts in a very simple form. BTW, I disagree with mtamada that was useless even though I totally agree there's a better way to do it and Z-scores was (and still is) in the queue. Also there's that...

Z-scores don't end up being tons different except that they weed out large disparity in players near the mean (since there are so many, ranking deviation is common).