Any Thoughts on Chad Ford's "Grantland" piece?

[Jeff Fogle]

Basic theme is:

"Front offices have turned to more quantitative methods of evaluating players. Spreadsheets and sophisticated statistical formulas play a much bigger role. Synergy's online video scouting service has revolutionized film sessions giving us the ability to watch video of every second that a prospect has played."

"a toxic combination of information overload and a dearth of access" has led to drafting getting worse.

http://www.grantland.com/story/_/id/668 ... -nba-draft

"Between 1990 and 1999, 37 percent of all lottery picks turned out to be All-Stars. 31 percent were busts. This decade, just 21 percent of all lottery picks have turned into All-Stars, while a whopping 42 percent are draft busts."

"In the '90s a typical draft produced 5.6 sleepers in the late-first or second round. These sleepers weren't necessarily All-Stars, but all were key NBA starters or rotation players. All were better than the draft busts in the lottery in a given year, which essentially means teams in the lottery missed on them. This decade? That number jumped to 8.3 sleepers per draft. What it means is that in the past 10 years, despite all of the advancements made in scouting, teams were more prone to miss (both ways) in the lottery than they were in the '90s."

Possible alternative explanations I was considering:

*Talent has flattened out in a way that presents more "maybe's" but hardy any "definitely's." So, the data isn't about the caliber of information, but the make-up of the talent pool. Are there a lot of undrafted guys on the All-Star teams? Meaning, teams were missing out on drafting all-stars and those players got gobbled up some other way? Or, were there just not a lot of all-star caliber guys out there waiting to be drafted?

*The increased emphasis over time on three-pointers at the middle school, high school and college level has created a lot of very similar players, very few of which would have the versaitility to be a franchise-changing star. (Or, also from the grouchy old guy perspective, we're going through a recent generation of youngsters who grew up watching Iverson and Kobe...and good NBA defenses at least have figured out ways to give that style some headaches, particularly if you're not as good as Iverson or Kobe were---inefficient scorers are more likely to be busts than superstars).

*The differering standards between superstars and sleepers is such that it's easier for a smart successful team to find somebody who fits their needs than it is for a struggling team to find a superstar. And, the flattening talent pool over the last decade has played into that.

*Ford used the term "crapshoot," but it's more like roulette. There are a lot of numbers on the wheel with very similar chances of what their career impact is going to be...and, no matter how much research you do you still can't know where the little white ball is going to land? You can't get lucky on purpose.

*No matter how much you know about earthquakes, you can't know where the next one is going to hit and how strong it's going to be. Are there elements of forecasting careers that are SO unpredictable that no amount of research is going to get you what you need to know? The handful of generational superstars are obvious. Beyond that, it's roulette and some players are going to work out while others don't?

Figured I'd toss this out for discussion if anyone was interested. I know a lot of you have been evaluating college and pro players in recent years. Do you feel your evaluations made you less able to predict? Or, has the sample size been limited in superstar potential to begin with?

[Crow]

I liked that he opened the topic but I have some questions and concerns from there.

"a toxic combination of information overload and a dearth of access" has led to drafting getting worse."

That's pretty sweeping. I'm not sold on the basic storyline as being sufficient or right or demonstrated right based on this article.

I am not fond of using # of All-Stars as a key criteria given the politics of All-Star voting in general and generational politics within that for which players got named repeatedly and which just missed. It is possible that the All-Star politics of the two decades are not that similar. I haven't reviewed that detail. I am not comfortable assuming they are equal. I'd prefer use of an objective measure for good player performance. WinShares or PER or whatever to avoid the All-Star issue altogether.

I like the idea of looking at busts but what is the standard? Objective and stand-alone or partly based on pick level and expectations of pick level? What does it take to qualify as a sleeper? I trust that he has thought carefully about these things but I'd rather know what standards he applied to evaluate his presentations about failures and surprises successes.

More "misses" wouldn't necessarily support a simple answer of worsening team performance. Maybe teams started taking more risky shots and maybe that might make sense for some or many of the also-rans or the teams who could afford to do it without needing it to work as much as others. Are you trying to win a bit more and more or win the title or who cares that much about the levels of not winning it? Those different goals might different draft strategies.

And I'd likely to hear about teams or types of teams who were doing well or poorly now or then, which teams improved or got worse and if the teams with an "analytic" approach / staff or simply more preparation did better or worse than average.

[Mike G]

"Between 1990 and 1999, 37 percent of all lottery picks turned out to be All-Stars. 31 percent were busts. This decade, just 21 percent of all lottery picks have turned into All-Stars, while a whopping 42 percent are draft busts."

This reads like some kind of joke. Of course, guys with 11-20 years in the league are more likely to have had an all-star season than those with 1-10 years.

Or did the author look at players drafted between 1990 and 1999 who had become all-stars by 1999?

How is 'bust' defined?
I'd guess that busts are less likely with the mandatory year between high school and drafting.

[schtevie]

Mike, exact.

But it is a bit worse still, I think. The fact is that as the league expanded over the time period, the chance for lottery "failure" did as well, so apples are being compared to oranges in this dimension too.

But anyway, I decided to check some numbers so Chad Ford wouldn't have to, and it turns out that there hasn't been any deterioration in drafting ability across the nineties and naughts. Remarkable consistency apparently. I looked at all players who had more than one All-Star game in their careers (so far). Of those drafted so that their first playing season fell in the 90s (i.e. 1989-90 to 1990-99), 76% of such All-Stars were lottery picks. Then weighting by All-Star games played, the figure rises to 84%. For the naughtie draftees, the comparable figures are 78% and 86%.

So, if anything, if Ford's thesis is true that access has in fact been restricted, so as to have made player evaluation more difficult, these numbers suggest that, all else equal, drafting performance has improved.

Then in the for "what it's worth" department (not much, for the aggregation issues and everything else) and assuming 24 All-Stars per year, in the 00s, draftees of the 90s (with two All-Star career appearances) occupied 64% of the All-Star rosters and draftees of the 00s (again with two career appearances to date) filled in 28%. (The 8% remainder being taken by as of now one hit wonders and those of an older generation.) So, yeah, All-Star games are typically and expectedly for those in the prime of their careers. And this point is underscored by a generational comparison, where the 90s class (again, who ultimately have had two or more appearances) had only occupied 13% of the All-Star roster in the 90s.

[Statman]

"Between 1990 and 1999, 37 percent of all lottery picks turned out to be All-Stars. 31 percent were busts. This decade, just 21 percent of all lottery picks have turned into All-Stars, while a whopping 42 percent are draft busts."

This reads like some kind of joke. Of course, guys with 11-20 years in the league are more likely to have had an all-star season than those with 1-10 years.

Or did the author look at players drafted between 1990 and 1999 who had become all-stars by 1999?

How is 'bust' defined?
I'd guess that busts are less likely with the mandatory year between high school and drafting.

Completely agree - I'm not sold on the analysis.

[Statman]

Mike, exact.

But it is a bit worse still, I think. The fact is that as the league expanded over the time period, the chance for lottery "failure" did as well, so apples are being compared to oranges in this dimension too.

But anyway, I decided to check some numbers so Chad Ford wouldn't have to, and it turns out that there hasn't been any deterioration in drafting ability across the nineties and naughts. Remarkable consistency apparently. I looked at all players who had more than one All-Star game in their careers (so far). Of those drafted so that their first playing season fell in the 90s (i.e. 1989-90 to 1990-99), 76% of such All-Stars were lottery picks. Then weighting by All-Star games played, the figure rises to 84%. For the naughtie draftees, the comparable figures are 78% and 86%.

So, if anything, if Ford's thesis is true that access has in fact been restricted, so as to have made player evaluation more difficult, these numbers suggest that, all else equal, drafting performance has improved.

Then in the for "what it's worth" department (not much, for the aggregation issues and everything else) and assuming 24 All-Stars per year, in the 00s, draftees of the 90s (with two All-Star career appearances) occupied 64% of the All-Star rosters and draftees of the 00s (again with two career appearances to date) filled in 28%. (The 8% remainder being taken by as of now one hit wonders and those of an older generation.) So, yeah, All-Star games are typically and expectedly for those in the prime of their careers. And this point is underscored by a generational comparison, where the 90s class (again, who ultimately have had two or more appearances) had only occupied 13% of the All-Star roster in the 90s.

Nice - now I don't have to look this stuff up. An article was written on a bad premise with no actual supporting evidence.

[wilq]

Basic theme is:
"a toxic combination of information overload and a dearth of access" has led to drafting getting worse.
[...]
"Between 1990 and 1999, 37 percent of all lottery picks turned out to be All-Stars. 31 percent were busts. This decade, just 21 percent of all lottery picks have turned into All-Stars, while a whopping 42 percent are draft busts."

IMHO he missed two giant points:
1) influx of high schoolers and international players!
In the early 90s there weren't many [if any] and then in the next decade number of high-schoolers exploded [shattering busts scale in the process] which was followed by a explosion of international players in the early 00s [adding another very risky group] so most of the 90s can look good in this environment.
2) Not all teams used quantitative methods!
There were busts... but who drafted them? IMHO that would be a great project: compare drafts of teams who jumped on stats bandwagon early and those who still drafted by watching games and workouts.

[Mike G]

Possible alternative explanations I was considering:

*Talent has flattened out ...
*The differering standards between superstars and sleepers is such that it's easier for a smart successful team to find somebody who fits their needs than it is for a struggling team to find a superstar. And, the flattening talent pool over the last decade has played into that.

Is anyone besides myself surprised to hear about the 'flattening' of recent new talent?
I haven't seen it in rookie eWins. Some yearly fluctuation, but not a steady decline.

You may see first-5-years Win Share totals here: http://bkref.com/tiny/EeWN7

I sorted these by season and find that for the top 100 first-5-years since 1990:
- Top players from 1991-94 thru 2000-04 averaged 184 WS per draft year, after prorating 1999 to 82 game equivalents.
- Top players from 2001-05 thru 2007-11 (last full 5-yr interval) also averaged 184 WS per draft year.

It's hardly surprising that the current decade isn't dominated by players from the previous decade.
From that b-r.com page, I see 9 of the top 20 commencing in this decade, with only 7 qualifying years.

[Jeff Fogle]

Thanks to everyone who commented. Interesting topic I think. Agree with skepticism about the thesis that the increase in work is leading to less successful drafts, or that drafts are even less successful currently when measured in context. A large amount of luck involved however you slice it.

Mike G., wasn't suggesting that it's a provable fact that talent has flattened out. Just throwing it out there as a possible alternative explanation that could explain Ford's premise (which is a disputed premise to begin with...from a guy who did amazingly comprehensive work in tracking down what teams where thinking about doing heading into the draft...don't want anyone to think I'm anti-Chad Ford or anything...just not sure about extra research leading to worse results, or the premise that results have been worse).

It's something you hear anecdotally. There are very few sure things in the draft, with a lot of "possible" stars who may or may not pan out. There's not much difference between the back end of NBA rosters and the D-League in terms of being able to contribute. If you gave a D-League guy 10-15 minutes a game, he could compile similar stats to the bench guys who are already getting 10-15 minute a game. Stuff like that.

Imagine a world where a grade of 90 or better is an All-Star, and 83 or better is role player. If the talent pool available for drafts doesn't have any 90's, but has a zillion US and foreign players who grade out in the 80-85 range...then the lottery teams will automatically be picking players who can't live up to what a high lottery pick has traditionally done in the past...while the later picks will provide role players for smart teams. And, the back ends of NBA rosters and all of the D-League will be those 80-85 range guys too over time. I think THAT kind of scenario would also explain the data Ford was presenting. If it's true that the number of all-stars was going down, and the number of "sleepers" was going up...it could be a flattening out of talent pool rather than "overthinking" the process.

Agree that the premise is disputed, and possibly disproved with the data presented in this thread. And, I'm far from married to the "flattening out" theory. Just presenting it as an alternative that does at least encompass some of the things you hear from people involved in the draft. There are few sure things. There are a lot of gambles.

Increasing research can't turn a talent pool of 85's into 90's. It's not the fault of research if the draft pool isn't rich with probable stars.

[Mike G]

Using fairly strict standards of 30 WS and a WS/48 of .125 over their first 5 seasons, there have just been 46 players since 1991.
Showing first year in the league.

Code: Select all

91 Payton, Coleman
92 LJohnson, Mutombo
93 Shaq, Mourning
94 Penny, Kukoc
95 Hill, EJones
96 Sabonis, Garnett
97 RAllen, Iverson, Kobe
98 Duncan, McGrady
99 Dirk, Pierce, Vince, Peja
00 Marion, Brand, Francis
01
02 Gasol, Kirilenko, RJefferson, Parker
03 Yao, Manu, Amare
04 LeBron, Bosh, Wade
05 Dwight, Iguodala
06 Paul, DWilliams, DLee
07 Aldridge, Roy, Rondo
08 Durant (4 yrs)

Even with the drought of '01, there were plenty around that year.
Basically all are bona-fide all-stars.

[Crow]

By this standard I count 21 such guys who entered in the 90s (not including 1990, so in 9 years) and 22 in the 00s so far thru 2008 entrants (9 years). So no drop-off in production of this level of performance in their first 5 years of play.

[Jeff Fogle]

Can we put that data into words? Meaning...something like... "the evidence shows that there's usually going to be 2-4 all-star caliber players in every draft, and that's not changing over time. The 14 (currently) lottery teams try to figure out who those 2-4 guys are going to be. Everyone else is drafting from a pool of non-All-Stars to fill roles."

Or, even, "the number of all-stars being drafted isn't changing, but improved analytics has led to an increase in the number of sleepers who come in and make important contributions."

How would you, Mike G., or Crow, sum up the findings in a way that represents your thoughts on Ford's thesis? Let's assume for now that my "flattening talent pool" possibility is a non-starter that's been thrown out for lack of evidence. What's the best way to sum up the findings?

[Crow]

I am not ready to sum up all the findings & all my feelings about them yet.

But I'll put a few more things out there.

I think the more detailed college and international stats and availability of compiled video from Synergy and others makes it easier for more folks to do what draft experts and team scouts use to do in small numbers and then based on that gave rather exclusive judgments. Now you can check to see if you agree with the writeup pretty far if you care to. I think that is a good thing for GMs, media who are not draft specialists and fans.

I can see how a draft expert or scout would regret the reduction in availability of behind the scenes information. I can't judge how good that information is or how well it has been used. I have not been in position for that access or former access.

I can see how draft experts and scout might not like the availability of more hard & detailed data to either compete with or substitute for their work and possibly disagree with it. But I'd rather have both types of information than just one.

How much has each set of information been used in the past? Has there been any long-term studies of the quality of the mock drafts or predictions of public draft experts? Scouting type predictions vs number-based predictions?

We can't judge the quality of the scouts' advice directly, but teams can. And the scouts vs the advice from stat analysis and those recommendations vs the final management actions which may be based on one or both and other stuff. It might also be possible to broadly compare teams who relied on "the numbers" more than average vs those who relied on them less, if you were an insider with a good feel for that or at least as good a guess about that as anyone has.



As for "flattening talent pool" I decided to break the drafts up into 5 year groups and look at WinShares to date in their careers per 82 games for the top 40 draft picks in those periods (200 players in each set). Here is what I found:

Drafted 1990-94

Top guy 12.3

average of top 25th percentile of player population 5.9
('WS per 82 games' for 'the median guy of the quartile' on career WS is 5.1)

avg of 25-50th percentile 2.9
(WS per 82 games for median guy 3.5)

avg of 50-75th percentile 1.1
(median guy 1.7)

avg of 75-100th percentile 0
(median guy -0.2)


Drafted 1995-99

Top guy 12.0

average of top 25th percentile of player population 7.5
(median guy 7.0)

avg of 25-50th percentile 3.3
(median guy 3.4)

avg of 50-75th percentile 1.4
(median guy 1.5)

avg of 75-100th percentile -0.3
(median guy -0.5)


Drafted 2000-04

Top guy 15.5

average of top 25th percentile of player population 6.8
(median guy 6.0)

avg of 25-50th percentile 3.0
(median guy 2.2)

avg of 50-75th percentile 1.4
(median guy 1.7)

avg of 75-100th percentile -0.2
(median guy -1.4, an usual guy I guess)


Drafted 2005-09

Top guy 14.7

average of top 25th percentile of player population 5.5
(median guy 3.6)

avg of 25-50th percentile 3.2
(median guy 2.2)

avg of 50-75th percentile 1.6
(median guy 1.1)

avg of 75-100th percentile 0.1
(median guy 0.0)


The guys in the last two timeperiods are earlier and much earlier in their careers than the first two time periods and that has some unknown at the moment impact on the numbers.

The average of the top 25 percentile has moved around some in both directions (up then down twice) but the rest of the top 40 draft pick quartile blocks have been quite stable. Not sure immediately what to conclude from that yet, but I'll put that data out there. Maybe Chad and Jeff are both drawing some their perspectives from this top 25 percentile "trend" in some degree.

I guess it would probably help to know the average draft ranks for these quartiles and the distributions thru the time periods. I'll probably look into that in a bit.

[Jeff Fogle]

Thanks VERY much for all that tabulating work crown (and in delayed form to Mike G for his notes earlier in the thread). Whenever I see the word "average," I have an anal-retentive knee jerk need to ask if the medians show anything different. Is it possible crow to tell by eyeballing it if you have it on spreadsheet to see if that makes a difference or not? From a distance, I'm wondering if there's something like...

0 players in 75-100th percentile
2 players in the 50-75th percentile
4 players in the 25-50th percentile

....who have meaningful performance (or something in that ratio at higher numbers) in a way that creates the averages you see...but the midpoint players of all three ranges are all pretty close to zero or negative one or something in those three hunks. May not be the case. And, please don't bother with it if it's not convenient to figure out. I've scooted down some wrong paths before because of averages so I was wondering if the medians confirmed the averages here or were slightly different.

The consistency of the averages you showed crow was very interesting...

[Crow]

Jeff, I addressed quartiles in part because of your stated view about the insufficiency of means. I also went back and added means and medians above. Anything else are you suggesting / seeking?


Mean draft pick # and # in top 15 picks by career performance quartile and by time period

Code: Select all

	early 90s	
	avg pick in top15
0-25%	10.7	36
25-50%	18.6	22
50-75%	24.9	8
75-100%	27.8	8


           late 90s	
	avg pick in top15
0-25%	9.5	40
25-50%	18.4	21
50-75%	23.9	12
75-100%	30.2	3


	early 00s	
	avg pick in top15
0-25%	13.7	29
25-50%	19.4	22
50-75%	20.0	17
75-100%	28.9	6


 	late 00s	
	avg pick in top15
0-25%	15.0	29
25-50%	18.3	19
50-75%	22.8	16
75-100%	25.9	11

The frequency and degree of early-entry across the time periods (as already mentioned by wiLQ) is another compounding factor that may partially impact / explain some or all of these trends. It may be unwise to attribute all draft pick accuracy change to shifts in draft pick access and greater use of data analysis.