Referreed Journals

[Guy]

The position adjustment is basically a kludge to smooth over the rebounding problem.

I suppose that's a much more succinct way of saying it!

[mystic]

This is a long discussion. But my quick take is that the problem is not the position adjustment per se (which improves WP). It's that the need for a position adjustment tells us there's a problem with the underlying productivity metric (adjP48). The metric says that guards are only half as productive as big men. Berri then says "well, teams need both big men and guards, so I'll make them equal in value to the big men." But why does a team "need" a SG, or a PG, or any particular mix of positions? It doesn't say that in the rules. So if it's true, it must be because a SG is delivering some kind of essential value to the team that isn't measures, or is mismeasured, by adjP48. How else can one justify saying that .400 adjP48(PF) = .200 adjP48(SG)? [There is also no logical connection between Berri's claim that teams "need" different types of players, and his conclusion that all positions are equal in value.]

When Berri initially got his productivity results, he had two reasonable choices: 1) believe in his metric, and say that big men are twice as valuable as guards, or 2) believe that positions are equal, and go back to the drawing board to figure out why his productivity measure claimed otherwise. Instead he took door #3: just define the positions as equal, with no empirical or theoretical justification, and then keep using the metric as is. But this creates a basic mathematial fallacy. Berri tells us that .400 adjP48(PF) = .200 adjP48 (SG), but also that .600 adjP48(PF) = .400 adjP48 (SG). And that does not compute....

Great post, I completely agree with that.

[xkonk]

Mystic, didn't you say just a few posts back that you wouldn't be surprised if many metrics thought that big players are more productive than small players? Guy's comments would seem to disagree with that.

Guy, is it then established that the five positions are generally equally productive? And do you disagree with the idea that teams need players who can pass and run the offense, and players who can rebound and protect the rim, and that these needs are roughly (and I'm happy to admit 'roughly') captured by position labels? And which part of my thought argument do you disagree with? It should apply to any metric you were building, and has nothing to do with wp.

[EvanZ]

Imagine that we invented a metric that only accounted for assists and points scored. Rebounding was determined to be irrelevant. In this scenario, point guards would have tremendous value. One could then either choose (A) to believe that a team should play 5 point guards, or alternatively, (B) one could normalize the rating by position. A small forward who averages 3 assists might then be considered as valuable as a point guard who averages 8 assists.

Of course, neither of these options is correct, because the underlying metric is flawed. Berri saw that "unadjusted" WP produced case (A), and then decided to remedy it by implementing choice (B).

[Guy]

Guy, is it then established that the five positions are generally equally productive? And do you disagree with the idea that teams need players who can pass and run the offense, and players who can rebound and protect the rim

I agree that teams need a mix of skills. But even if something like the 5 "positions" represents the ideal mix, it doesn't have to follow that all 5 positions are equally valuable. What would be true is that replacement level players at each position have equal value, i.e. zero. But the average value per position will depend on the variance of skill at each position. If some positions have more variance, then they will have more value.

But it's not obvious to me that there are really five distinct positions, each of which a team "has" to have. For example, is a team better off with the 30th best PF in the league than having a second C of average ability? That seems unlikely ....

[mystic]

Mystic, didn't you say just a few posts back that you wouldn't be surprised if many metrics thought that big players are more productive than small players? Guy's comments would seem to disagree with that.

Where does Guy's post disagree with that position? Are you aware of the fact that 1.1 is also bigger than 1.0? When I say that I wouldn't be surprised, if basically all metrics come up with the conclusion that overall bigger players are more "productive", I mean a much smaller difference than what Berri arrived at.

And do you disagree with the idea that teams need players who can pass and run the offense, and players who can rebound and protect the rim, and that these needs are roughly (and I'm happy to admit 'roughly') captured by position labels?

See, and that is the issue here. It is not necessarily the case, that a player with a certain position assigned to plays that on both sides of the court. How do you want to handle players, who are defending the point guards on defense, but are playing more like an off guard or small forward on offense? Is a C who can shoot from the outside really playing the C position on offense? And the most important thing: Do you believe that an assist or a point scored is less valuable, because a PF instead of a SF has those things in the individual boxscore listed?

Just answer the following questions honestly: Would you have taken Troy Murphy in 2009 or 2010 over Dirk Nowitzki? Do you believe that Troy Murphy would have lead the Dallas Mavericks to more wins instead of Dirk Nowitzki? Do you honestly think that the Dallas Mavericks with Kris Humphries in place of Dirk Nowitzki would have won more games last season? And why are the Knicks constantly getting outscored big time with Tyson Chandler on the court? He was supposed to be the biggest impact factor on the Mavericks according to WP48, now his WP48 has even increased (from 0.268 to 0.358) and yet the Knicks are playing far worse with Chandler on the court compared to without him. Do you agree with WP that Kevin Love massively declined from last season to this season (going from 0.342 to 0.244 in WP48)?

[mtamada]

Eigenfactor is a newer metric that is probably an improvement, but much less often used:

http://www.eigenfactor.org/methods.php

Cool idea. Sort of the Adjusted Plus Minus of Influence Factor measures. How good do the results look? APM is great in theory but turns out to have high standard errors (low reliability) in practice. I'd imagine that EigenFactor, much as with player valuation techniques, has good agreement on what the superstar journals are -- it's easy to recognize that LeBron James is a great player or that the AER is one of the top economics journals. Where this gets really interesting is when we look at the second-tier and third-tier players and journals. That's where different models might come up with very different ratings -- and also where self-citation might be more of an issue, and it becomes more important to figure out how to handle it correctly (if indeed there is an all-encompassing correct answer).

[EvanZ]

Well, since you seem to know something about the economics field, take a look at this list:

http://www.eigenfactor.org/rankings.php ... igenfactor

How does it look?

[xkonk]

Just answer the following questions honestly: Would you have taken Troy Murphy in 2009 or 2010 over Dirk Nowitzki? Do you believe that Troy Murphy would have lead the Dallas Mavericks to more wins instead of Dirk Nowitzki? Do you honestly think that the Dallas Mavericks with Kris Humphries in place of Dirk Nowitzki would have won more games last season? And why are the Knicks constantly getting outscored big time with Tyson Chandler on the court? He was supposed to be the biggest impact factor on the Mavericks according to WP48, now his WP48 has even increased (from 0.268 to 0.358) and yet the Knicks are playing far worse with Chandler on the court compared to without him. Do you agree with WP that Kevin Love massively declined from last season to this season (going from 0.342 to 0.244 in WP48)?

I don't take WP as my bible if that's what this question is asking. I'm also not sure what this has to do with position adjustments. Is there a way to have a discussion on this forum about something even tangentially related to WP without it becoming a discussion about WP?

I'll try again. Do we think there are a variety of skills that a team needs to have, and are those skills differentially spread across players? If a team needs to have a guy who can bring the ball up court, it really doesn't matter how productive he is compared to all the guys who don't have that responsibility, right? It matters how good he is compared to all the guys that bring the ball up for other teams. Guy says that he doesn't think such a thing maps on to five distinct positions, which seems reasonable. Are there potentially different position 'prototypes' that would do better, and players would be a combination of some of them? Maybe Kevin Love is a 'post offense' and 'floor spreader' while Tyson Chandler is a 'rim protector'. Or can teams be viably constructed in so many ways that there is no universally required skill?

[mystic]

If a team needs to have a guy who can bring the ball up court, it really doesn't matter how productive he is compared to all the guys who don't have that responsibility, right? It matters how good he is compared to all the guys that bring the ball up for other teams.

Correct. But how valuable is such a trait in comparison to other skills? Implying that each of those skills is equally important and valuable is the mistake here. Different skills have different value, equalizing them will give you likely a wrong impression about the value of a player.

[Guy]

If a team needs to have a guy who can bring the ball up court, it really doesn't matter how productive he is compared to all the guys who don't have that responsibility, right? It matters how good he is compared to all the guys that bring the ball up for other teams.

If you can establish that all teams have a player with this responsibility, I would agree it seems reasonable to conclude such players have value that is currently showing up in the statistics of other players (reducing their turnovers, increasing their efficiency). But as Mystic says, it doesn't necessarily follow that players who do this are equally as valuable as all other roles/positions. What I think you could say is that the worst NBA player in this position (i.e. a replacement level player) has the same value as the worst player in another position.

BTW, one way to infer the value of a role you feel is poorly measured by current stats is to identify players who play that role, and then see how much worse they are at the things you can measure. In baseball, for example (which I know much more about), analysts do this to measure the value of defensive performance at a given position. We don't really know how to compare the average defensive value of catchers as compared to first basemen (only how to compare players within a given position). So we compare their hitting, and find that catchers hit much worse than first basemen, something on the order of 2-3 wins a season. We then infer that catcher defensive skills are worth that much more than the defensive skills of an average firstbasemen, since teams are willing to accept that much less offensive production to get it. I don't know if a similar approach could be helpful in basketball, but maybe so.

[mtamada]

If a team needs to have a guy who can bring the ball up court, it really doesn't matter how productive he is compared to all the guys who don't have that responsibility, right? It matters how good he is compared to all the guys that bring the ball up for other teams.

Correct. But how valuable is such a trait in comparison to other skills? Implying that each of those skills is equally important and valuable is the mistake here. Different skills have different value, equalizing them will give you likely a wrong impression about the value of a player.

Yes, that's the key. All positions (and all or almost all skills) are important -- but they are almost certainly not equally important (and their importance probably varies by roster, opponent, and matchup). An analogy: just as teams need to have somebody bring the ball up and get the ball to the big men inside, teams need to have a backup center. (Even Wilt was on the bench for a few minutes in 1962-62.) The backup center is in that sense a vital and necessary part of the team. You have to have one, and the better he is, the better off your team is.

But he's clearly less important than the starter. When it comes to evaluating teams' overall talent levels, the backup center is just plain less important. But clumsy position adjustments can artificially inflate such players' estimated skills or contributions.

Note: I'm not claiming that "backup center" is a position in the same sense that "point guard" and "center" are positions. The point is that it's a mistake to assume that all positions are equally important just because teams have to have five players on the court and always have a point guard (or PG equivalent e.g. a point forward).

I think position adjustments can have value (e.g. comparing point guards with each other, rather than with all players), but they have to be done carefully.

[mtamada]

Well, since you seem to know something about the economics field, take a look at this list:

http://www.eigenfactor.org/rankings.php ... igenfactor

How does it look?

Fascinating, more differences amongst the top ranks than I was expecting. My superficial reaction is that both had surprises but EF looked a bit better than AI. But what really jumped out was how some journals with very different audiences or breadth were mixed in amongst the top ranks. A number of finance journals e.g. Finance has been a field of ever-growing importance (recognized with several Nobel prizes awarded to financial economists over the years) but it was still surprising to see journals which are presumably of interest to a narrower, smaller set of economists outrank journals with wider broader appeal. I'm not saying that's wrong, just surprising.

And the high ranking of the Journal of Economic Perspectives surprised me but perhaps it shouldn't have. It's aimed at a general-purpose audience, including upperclass undergraduates in some cases. For sure it's widely read or at least skimmed be economists ... widely cited too evidently as it ranks high under both measures (but outside of the Top 10, so maybe that's about right).

The J of Economic Literature is another unique journal, it's all review or survey articles. It's #3 in AI but #20 in EF. Really hard to say which seems like a better ranking, because the articles there have a special purpose, as do the J of Econ Perspectives'.

With economics having those specific sub-categories such as finance, and specialized journals such as JEP and JEL, maybe what the Impact Factors need is a "position adjustment" just as basketball has different positions.

[EvanZ]

With economics having those specific sub-categories such as finance, and specialized journals such as JEP and JEL, maybe what the Impact Factors need is a "position adjustment" just as basketball has different positions.