Draft projection models

[VJL]

The data goes back to 2004, which isn't as far as I like, but is as far back as KenPom's numbers go. Is that too small of a dataset? I've been using DraftExpress for combine measurements. In our regression, the combine numbers did make a significant impact. Another thing that made a large impact was separating into two different groups by height and running separate regressions. The bigs performed much better than the smalls.

No I don't think it is too small. Before I started using the BBR dataset I was working with the DX dataset which goes back to 2013 and getting reasonable results. My one caution would be how many datapoints you are losing with the combine data. Most guys have height, weight, wingspan, and vertical tested (though definitely not all... and many stars sit that one out), but if you start including things like sprint, agility, and bench, you begin to lose a lot of players.

Sounds like a good way to discriminate by position. Especially with only back to 2004 I would be uncomfortable running a different regression for each position. I decided not to do that with mine either since some guys fill roles that are inconsistent with their position but do so effectively, and I hate to see them get punished for it. My model above just includes position as one of the terms. That method seems to work better than I thought it would.

Today I played with integrating combine data as a post-hoc adjustment to the players for whom that data is available. The only measurement that added any information was no-step vertical.

[colts18]

What combine stats did you find have a positive correlation to success?

[VJL]

What combine stats did you find have a positive correlation to success?

When I was including them alongside the boxscore statistics, no step vertical and standing reach were the only predictors worth including.

Now that I tried applying combine stats post-hoc for the newer data (including it in a regression along with the predictions posted in the parent post such that -- [Observed Wins ~ Predicted Wins + No step vert + Standing reach] I found some interesting things:

1) Standing reach is now a negative. What that tells me is that while standing reach is obviously important to playing basketball, if you can't use it to record rebounds/blocks/points... in college, you won't figure out how to use it in the pros and some of the stats you got just by being bigger than anyone else will be tougher to come by.
2) No step vert is still informative, but the effect was really weak until I included an interaction term between predicted wins and no step vert. Basically, if you don't look like a good prospect based on the box score stats it doesn't matter how high you can jump. However, as your expected production increases, being able to jump out the building becomes an increasingly important predictor of your ceiling.

[Mike G]

... being able to jump out the building becomes an increasingly important predictor of your ceiling.

Love a good mixed metaphor.

[AcrossTheCourt]

Did you use the player's listed height in any of those models? Did you use it with the combine measurements?

[VJL]

I used listed height and weight in the "likelihood of bust, bench, starter, star" model, and in the player comparisons, but I did not use them in the expected win peak model (because they didn't add much information). I have since added height in weight back in to the wins model... and I should probably post the new results....


I found that when using a dataset with combine info, height falls out as useful when you add standing reach. This is consistent with the idea that you don't block/alter shots with your head.

[VJL]

UPDATE:

I did a little more tinkering.

First was something I have been meaning to do for awhile. I raised my bust threshold from 0 win shares to about 1. This adjustment captures all of those players who managed just enough miserable minutes to eclipse 0 but ultimately failed as busts. The result is that the percentages are more smoothly binned across likelihoods rather than having a ton of “bench” players which was bothering me.

Second, I abandoned the idea of trying to adjust the WS model for combine measures post-hoc. Instead, I made an entirely new set of models in addition. I now have the Win Shares-based model that uses basic box scores, strength of schedule, age, and size (new) and draws from a dataset beginning in 1982. I also have a new RAPM-based model that uses not just all of the above information but also combine measurements but only draws from a dataset beginning in 2001.

The mean and variance are both larger in the RAPM-WARP-based models so you can’t really compare across them other than relative rankings. I will probably play with these a little bit more before the draft, but they are pretty close to where they will be.

[AcrossTheCourt]

How a player translates his college play to the NBA is controlled by his environment. Have you considered adding a measure for player development? For example, getting drafted to the Spurs probably helps you reach your potential better than, say, the Bobcats. If there's one team who could lead to their own variable, it's the Spurs.

[VJL]

How a player translates his college play to the NBA is controlled by his environment. Have you considered adding a measure for player development? For example, getting drafted to the Spurs probably helps you reach your potential better than, say, the Bobcats. If there's one team who could lead to their own variable, it's the Spurs.

That could potentially be a cool addition. I'm not sure what the best way to operationalize "environmental goodness" is. Team record in previous year? It would be especially difficult, because I wager most of the "environmental" importance is idiosyncratic. What is a great situation for one guy may be a terrible one for another depending on how skills mesh with needs.

[Mike G]

Agree with that. Some kids need to be brought along slowly, and some just don't have the patience .
A liability to one player may be an opportunity to another type.

[mtamada]

Right, although it'd be great to measure developmental success, it'd be devilishly difficult. Even forgetting about development, just measuring plain old fit between a player and team is hard. Jeremy Lin was on the verge of washing out of the league entirely. Then he looked like an all-star, and now he looks like a marginal starter. Slick Watts was a good fit on some middling-to-bad Supersonic teams -- and a very poor fit basically everywhere else. When the Sonics replaced their coach early in the 1978-79 season and became a title contender, Slick not only lost his starting position, he was no longer used as a bench player either. The Jazz and Rockets couldn't make use of him either and within two years he was out of the league.

[bchaikin]

Slick Watts was a good fit on some middling-to-bad Supersonic teams -- and a very poor fit basically everywhere else. When the Sonics replaced their coach early in the 1978-79 season and became a title contender, Slick not only lost his starting position, he was no longer used as a bench player either. The Jazz and Rockets couldn't make use of him either and within two years he was out of the league.

slick watts - you are correct (he was out of the league 2 years later) but not a good example nonetheless...

first off, in the decade of the 1970s (69-70 to 78-79), the chances of a player being in the league at an older age was far different than in the 2000s (99-00 to 08-09):

age..1970s..2000s
23....100%...100%
28.....48%....67%
30.....33%....51%
32.....21%....46%

less than 1/2 of the players in the league at the age of 23 in the 1970s were in the league by the age of 28. in the 2000s it was 2/3. just 1/3 of the players in the league in the 1970s at the age of 23 were in the league at age 30, but in the 2000s it was 1/2. guaranteed contracts and better recovery from injuries (better medical practices) are the key reasons for that...

as for watts he was one of the league's top PGs in the mid-1970s - from 74-75 to 76-77 he had more assists and far more steals than any PG in the league (playing just 31 min/g mind you). he was as much of a "star" in the league as a player could be not playing in a major city - people loved the quick little bald guy with the headband and kneepads...

in 77-78 not only watts but the sonics starting SG freddie brown, who was one of the league's top scoring SGs (20.5 pts/g the 3 previous years), was demoted too, to 1st guard off the bench, as seattle started dennis johnson (future hall-of-famer) and gus williams (soon to be one of the league's top guards for the next half decade). watts was subsequently traded in 1/1978 to the jazz - for a 1st round pick...

watts could have fit in with any number of teams at that time, but in new orleans they had pete maravich and gail goodrich (who i believe at the time were the two highest paid players on the jazz and were not going to be replaced by anyone as both were drawing cards), and when maravich went down with injury 2/3 of the way through the season they started james mcelroy as he knew their system. but for the rest of that season watts was the 1st guard off the bench for the jazz and played 20 min/g in 39 games...

the next season watts was traded (9/1978) to houston - again for a 1st round pick, because when the rockets signed rick barry as a free agent they lost starting PG john lucas as compensation. on that team barry at SF was the primary ball-handler, but at guard they had super-small calvin murphy to score (just 5-10 in height) and the 6-7 robert reid to play defense, and a 24 year old backup PG named mike dunleavy who shot the lights out (2nd best shooting PG in the league that year). so despite trading for him they had no need for a small high assists PG...

but watts' stats those two seasons for the jazz and rockets were very similar to what he did for seattle - high assists and high steals. among the 46 PGs that played at least 1000 minutes those 2 seasons (77-78 and 78-79) watts had the 8th highest ast/min rate and the 8th highest steal/min rate. a number of teams back then could have used that - he just happened to be traded to two teams that could not use a high assists but small PG at that time...

was watts any worse back then than mike bratz, lloyd walton, john kuester, henry bibby, ricky green, eddie jordan, charlie criss, and others? he couldn't shoot but he did rebound, pass, and get steals. yet those PGs played as many or more minutes as he did...

and as for being out of the league by age 28, again just over 1/2 of the players back then that played in the league at age 23 were out of the league by age 28, 2/3 by the age of 30...

[mtamada]

That still leaves the question of why didn't some other team pick up Slick Watts. Teams found places on their roster for the likes of Bratz, Kuester, and Walton (Lloyd not Bill), but not Watts.

Watts in this sense was very different from the unknown, untried Jeremy Lin. He was one of the better known players in the league, a steals and assist leader ... exactly the kind of player, seemingly, that teams would think was worth a first-round draft choice.

And then ... nothing. Even if the Jazz and Rockets had no place in their rotation, why didn't some other team pick him up, especially after he became freely available and the price was no longer a first round draft choice?

The answer, I eventually realized (and Sonics fans had started to get an inkling of this even when he was still playing for the Sonics): Slick Watts was, for most teams, not a very good player. Even at the time, he was known for his harum-scarum, chaotic style of play. If a team is willing to put up with that, it could have an okay player and be a poor-to-middling team, as the Sonics were in the mid-1970s. If a team wanted a player who'd actually run an organized offense, there was no place for Slick Watts.

That's the importance of fit.

Watts was essentially the same player all along. He played hard, he got steals, he could penetrate and get assists. He did that from his rookie year (he even stood out in his first training camp with the Sonics) and he did that in his last years with the Sonics. If we look at just those stats, he's the kind of player that a lot of teams could've used, as a backup PG if nothing else.

Yet we have powerful evidence that teams could find no place for him. Not even the Sonics, after they became a good team and went away from his style of play. He was not some overlooked Jeremy Lin; by 1979 teams knew what Watts could do and he was freely available -- and they all said no thanks. Why? Because he could pick up some stats that looked good, but eventually the NBA realized that he could not help most teams win.

He's still a fan favorite in Seattle, one of their most memorable players ever. He was on the Sonic teams that made it into the playoffs for the first time in their history. He could and did contribute to those teams, and that's something. But for most NBA teams, he was not good enough to be on their roster.


Also: comparing career survival rates in the 1970s to the 2000s is skewed by the ABA. A lot of roster spots disappeared overnight when teams folded. And players could disappear from a league simply by jumping to the other league, causing the percent of players who remained in the NBA to be inherently lower. There's no question that we're seeing more players with long careers these days, but the direct comparison to the 1970s is affected by the existence of a rival league, not just inherent career length.

[Crow]

Model 3 (Comparison finder):
This is just a fun little model that helps find past player seasons that are similar to ego’s. All it does is look for the players who minimize the absolute difference in average standard deviation across a set of statistics. The actual math chosen was a bit arbitrary and someone may conjure a better version, but here is what I am using for now:

Code: Select all

((abs(2P.X – 2P.Y) + abs(2PA.X – 2PA.Y) + abs(3P.X – 3P.Y) + abs(3PA.X – 3PA.Y) + abs(FTA.X – FTA.Y) + abs(FT.X – FT.Y))/3 + 
(abs(AST.X – AST.Y) + abs(TOV.X – TOV.Y))/2 + 
(abs(STL.X – STL.Y) + abs(BLK.X – BLK.Y))/2 + 
abs(TRB.X – TRB.Y) + 
abs(PF.X – PF.Y)/4 +
(abs(Height.X – Height.Y) + abs(Weight.X – Weight.Y))/2 +
abs(Age.X – Age.Y) + 
(abs(SOS.X – SOS.Y) + abs(SRS.X – SRS.Y))/2)   
) / 7.25

This simple model can be suggestive and prompt further thoughts.

I just wanted to add that I'd ideally significantly prefer to variably weight the elements, based on other research and judgment. I can't give any comparison system (another example is Schoene) deep credence that does not attempt a thoughtful set of weights for the elements used in the model. It may not be possible to declare a weight system as "best" but there are better and worse and I believe there are plenty better possible than every element given equal weight, if I am understanding the models correctly.

[VJL]

I just wanted to add that I'd significantly prefer to variably weight the elements, based on other research and judgment.

Agreed. I don't worry much for the comps since it is more a scavenging tool that brings up interesting potential names that I can investigate further. One thing I have done recently with some success is add "predicted wins" generated from the other model as an additional factor. This at least indirectly weights the inputs to the comparison based on what they mean to expected NBA production. What I like about it is that it finds players who had both similar stat lines and were close enough in the important things to be viewed as comparable prospects. This helps identify skill clusters that are misidentified by the prediction model. For example, Kelly Olynyk does really well in the wins prediction model, but his comparison set is a bunch of guys who did well on the prediction model, but not in reality. Other cases like CJ Leslie show the reverse where similar low projected players ended up succeeding.

I posted these new comparison results and other more recent model outputs here http://www.canishoopus.com/2013/6/3/439 ... arter-bust.