APBRmetrics

The discussion of the analysis of basketball through objective evidence, especially basketball statistics.
It is currently Fri Oct 24, 2014 5:58 pm

All times are UTC




Post new topic Reply to topic  [ 95 posts ]  Go to page 1, 2, 3, 4, 5 ... 7  Next
Author Message
PostPosted: Thu Feb 06, 2014 5:45 pm 
Offline

Joined: Mon May 20, 2013 11:18 am
Posts: 51
I have a google-doc with a collection of draft models and predictions I wanted to share here.

Models include:

"EWP" model (http://bit.ly/1cuim4L) which uses a player's peak season in terms of combined Win Shares and RAPM-wins as the dependent variable. The model draws from NCAA prospects going back to the early 80s and uses random effects to address the contextual problems that this deep history creates.

"Humble" model (http://bit.ly/1lXF3hc) which is similar to the EWP model, but is heavily influenced by expert opinion (i.e. a transformed draft-order variable historically and DX + Chad Ford rankings for predictions)

"RAPM-DIF" model (http://bit.ly/MrwZKK) which uses RAPM-O and RAPM-D as the dependent variable of two separate models. This model draws from players going back to 2002.

"Star %" model (http://bit.ly/MrwZKK) which uses multinomial regression to peg a player's likelihood of being either a bust, bench-player, starter, or star drawing from NCAA prospect going back to the early 80s.

"ATH" model (http://bit.ly/1avXFDB) which won't have ratings for 2014 until after the combine, but tries to predict NBA production while using only physical data.

and finally "Statistical Comps" (http://bit.ly/1aEa6cC) which is just a simple metric that finds the average standard deviation difference between each prospect and historical players back to the early 80s across box-score statsitics, age, size, and team context and lists the most similar players it finds.

You will find a link to "out of sample" retrodictions for each of these models on the associated worksheet in the google-doc. I also recently added a list of the variables included in the model along with parameter estimates and standard-errors to the prediction page for the EWP, Humble, and RAPM-DIF models. I have put an emphasis on keeping these models simple both to avoid over-fitting as well as make it easier to understand why certain players look good/bad and subjectively assess where predictions may or may not be appropriate in a given case.

I tweet updates every Tuesday (https://twitter.com/VJL_bball) and am happy to take requests for players not in the lists (though some guys I may not want to add until the season is finished just to make updating easier.)


Last edited by VJL on Thu Feb 06, 2014 7:45 pm, edited 1 time in total.

Top
 Profile  
 
PostPosted: Thu Feb 06, 2014 7:24 pm 
Offline

Joined: Thu Apr 14, 2011 11:10 pm
Posts: 2426
I appreciate your sharing this work and have looked at it several times. I might have some comments and / or questions at a later time.


Top
 Profile  
 
PostPosted: Thu Feb 06, 2014 7:35 pm 
Offline

Joined: Mon May 20, 2013 11:18 am
Posts: 51
Crow:

Good to hear. I appreciated your comments last year.


Top
 Profile  
 
PostPosted: Thu Feb 06, 2014 7:58 pm 
Offline

Joined: Mon May 20, 2013 11:18 am
Posts: 51
I wanted to note that comparing the strength of different predictors between the EWP and Humble models is particularly interesting. Because the Humble model is controlling for draft position, this comparison gives a sense of what features NBA decision-makers are under/over-valuing when making selections. For instance, both SOS and MOV are negative predictors in the humble model in spite of being good positive predictors in the EWP model. This says that folks are probably too quick to knock a player for being in a lesser conference or a part of an underperforming team. Steals, rebounds, and shooting efficiency vs. volume look to be the traits that evaluators most underappreciate.


Top
 Profile  
 
PostPosted: Thu Feb 06, 2014 11:52 pm 
Offline

Joined: Sat Apr 16, 2011 7:40 am
Posts: 298
Location: Cambridge, MA
I love the comparisons. This is very very useful... Thanks for sharing!

_________________
http://pointsperpossession.com/
@PPPBasketball


Top
 Profile  
 
PostPosted: Fri Feb 07, 2014 5:23 pm 
Offline

Joined: Mon May 20, 2013 11:18 am
Posts: 51
Steals appear to be a great predictor of NBA success. This is probably the one insight from NCAA-to-NBA projections that has gained the most traction. One major skepticism about this relationship is how it applies to players in a zone-D system. I ran the full draft model with by-Coach random slopes on the effect of steals to try to investigate this concern. It looks like there might be good reason to worry that steals within the zone don't say the same thing as steals within a man defense:

Image


Top
 Profile  
 
PostPosted: Fri Feb 07, 2014 5:33 pm 
Offline

Joined: Fri Feb 07, 2014 5:28 pm
Posts: 6
Cool stuff. I had been operating under the assumption that steal rate for a team like Syracuse is worthless, nice to see it confirmed.

One of your model's projections that I can't wrap my mind around is Aaron Gordon projecting to have a +1.6 oRAPM. Can you elaborate on how that's even possible given his historically poor shooting numbers? I know some low TS guys improve considerably with age, but when accompanied with a 42.2% FT% it seems like it becomes a much less likely proposition.


Top
 Profile  
 
PostPosted: Fri Feb 07, 2014 5:57 pm 
Offline

Joined: Mon May 20, 2013 11:18 am
Posts: 51
I have trouble understanding Gordon's ORAPM projection as well. Part of it is that because eFG translates so terribly between leagues the offensive model doesn't have a ton to work with. That makes age really stand out. Add a good offensive rebound rate, decent assists, and probably more usage than he should be allowed and you get a solid rating. I don't really believe it though.

His top 8 comps (as of a couple weeks ago) are: Rudy Gay, Devin Ebanks, Luol Deng, Al-Farouq Aminu, Keith Langford, Michael Kidd-Gilchrist, Chucky Brown, Terry Cummings

So there are some offensively successful guys who posted similar numbers as kids, but I wouldn't say offense is anything close to the defining feature of that group.


Top
 Profile  
 
PostPosted: Fri Feb 07, 2014 7:35 pm 
Offline

Joined: Thu Apr 14, 2011 11:10 pm
Posts: 2426
Humble-EWP (“draft experts – analysis from the stats):

Biggest negative in top 10 on Humble within 2013 draft class? Anthony Bennett (9th on Humble, 17th on EWP but picked #1). Len second biggest difference. McElmore (11th on Humble) would have been the biggest (13 spot difference). Muhammad was 22nd on Humble, 33 spots higher than EWP ranking. Silva 12 spots higher on Humble. EWP looks better than Humble on these 5 extreme variance cases.

Most undervalued on Humble top 20 compared to EWP? Muscala, 20th on Humble, 6th on EWP but not in the NBA yet.

Steven Adams a standout on RAPM-Dif model and 8 to 11 points higher than on Humble or EWP.

Noel #1 on all three models.



Among 2012 draftees, Kidd-Gilchrist was rated 6 spots higher on Humble than EWP and that appears to be a moderate mistake again. 14th on RAPM-Dif. Both had Waiters at 3rd, RAPM_Dif was only slightly better (IMO) at 7th. Humble had Terrence Jones at 14, EWP at 12, RAM+Dif at 5th. Harrison Barnes 9th on Humble, 25th on EWP, 22 on RAPM-DIF.
Lilliard 6th on Humble, 19th on EWP and 19th on RAPM-Dif. IMO Humble was off on 4 of these 5.


Top
 Profile  
 
PostPosted: Fri Feb 07, 2014 8:06 pm 
Offline

Joined: Fri Feb 07, 2014 5:28 pm
Posts: 6
VJL wrote:
I have trouble understanding Gordon's ORAPM projection as well. Part of it is that because eFG translates so terribly between leagues the offensive model doesn't have a ton to work with. That makes age really stand out. Add a good offensive rebound rate, decent assists, and probably more usage than he should be allowed and you get a solid rating. I don't really believe it though.

His top 8 comps (as of a couple weeks ago) are: Rudy Gay, Devin Ebanks, Luol Deng, Al-Farouq Aminu, Keith Langford, Michael Kidd-Gilchrist, Chucky Brown, Terry Cummings

So there are some offensively successful guys who posted similar numbers as kids, but I wouldn't say offense is anything close to the defining feature of that group.


OK, that makes sense. He's such an outliery bad shooter for a wing prospect I imagine he's going to have a flukey output for most models. I doubt his 7 for 18 start shooting 3's helped your model appreciate the badness of his shooting either.

Do you adjust for steal rates based coach/team defensive strategy? For instance, coaches like Bill Self and John Calipari regularly have excellent eFG defenses that force few turnovers, whereas Shaka Smart is willing to sacrifice eFG to create a boatload of turnovers with his full court press. I suspect that adjusting for large discrepancies between team defensive eFG% and defensive TOV% can sharpen steal rate a bit beyond just zone vs man.


Top
 Profile  
 
PostPosted: Sat Feb 08, 2014 1:13 am 
Offline

Joined: Mon May 20, 2013 11:18 am
Posts: 51
"Do you adjust for steal rates based coach/team defensive strategy?"

No. However it is absolutely something I recommend doing when integrating the model outputs into more subjective player assessments. In addition to the plot I posted above where I show that STLs may not carry as much information for zone-D systems, I should note that Norm Stewart, Bill Self, and Bob Knight all famously man-to-man anti-gamble coaches are in the bottom eight on that same list among Westhead, Beilein and Pitino. I think this is showing the other end of the extreme. What it tells me is that, for example, I wouldn't recommend subjectively doubling the steal rate for a Kansas guy, but rather just give that particular statistic less attention overall if you think the system is warping the way plays do/do not get them.

In short... it is complicated, and I would rather leave things simple enough for folks to work from a more simplistic baseline than try to capture all of the nuance in a single model that would ultimately end up over-fit.


Top
 Profile  
 
PostPosted: Sat Feb 08, 2014 1:37 am 
Offline

Joined: Mon May 20, 2013 11:18 am
Posts: 51
Crow:

My sense in that the Humble does a good job of discriminating among the thicker group of "really good, but not great" statistical prospects and catching guys like Muscala who the numbers like but may never do much (either because they aren't particularly good or they never get a legitimate shot). However, I tend to take the EWP seriously when it thinks a highly touted player looks bad like with Muhammad and start to worry about guys like Bennett and Len on the fringe around 5 EWP.

Historically, the EWP catches quite a few notorious busts: Morrison (2.7), Olowakandi (4.3), Flynn (3.9), Wes Johnson (3.1), Mayo (3.2), Joe Alexander (1.7), Fizer (3.7).

The clearest case going the other direction, where the EWP really didn't like a high pick and that guy turned out to be a star is Deron Williams (3.4). There are some other fringe cases like Brandon Roy and Russell Westbrook, but at least those guys were predicted to be starters (5+) . This is bad news for Andrew Wiggins who at 4.9 is admittedly higher than the busts I just mentioned, but flirting with dangerous territory.

RE: Lillard

I want to use him as an example of the most common way guys seem to blast way ahead of their projected ratings. Shooting. Lillard basically hit the NBA court shooting better from range in the NBA than he did from the college line. Shooting is tough to predict generally... and it is also one skill that I am a true believe guys can simply improve at. We are seeing the same thing this season, but even more extreme with Tim Hardaway jr. The guy wasn't even a good college shooter and now he is hitting 40% on more than 7 attempts from 3 every 40 in the NBA. Michael Redd, whose major criticism going into the draft was his shooting, is a great historical example, but there are lots of other cases. The single best way to go from mediocre NCAA prospect to NBA star is to fix your shot.


Top
 Profile  
 
PostPosted: Sun Feb 09, 2014 8:26 am 
Offline

Joined: Thu Sep 20, 2012 6:44 pm
Posts: 80
VJL, do you use FT% and 3pt volume to judge shooting? Lillard is an interesting case because although "only" a 40% 3pt shooter, doing it while taking 7.2 3pt attempts a game and while hitting 89% FT to me screamed elite NBA shooter at the time.

But there is other guys where it's really hard to predict, especially considering it looks to me like the Rockets and Spurs are doing stuff converting shooters that other teams aren't. Kyle Lowry and Chandler Parsons are two guys where it'd be almost impossible to confidently predict how good of pros they've become. Lowry's 3pt shot was non-existent in college through 4(!) years of his career and now he's one of the best in the business at it. Parsons was hitting 3s a little in college but was 56% from the FT line his final season, nobody in the NBA touches 3pt range if they have a stroke like that at the line. Unless someone was a master of evaluating shot form and seeing hidden gems in Lowry and Parsons, I think the best you could do is liking them more than the NBA did, but calling them top 5 prospects in their draft at the time and then having this happen would probably just be plain luck. Still that far from makes drafting artless. If one can figure out one guy has a 10% chance of being as good as Parsons does now and the other guy an 80% chance there's clearly value in that. In addition you give Victor Oladipo the same shooting surge Parsons has had and instead of becoming a very good player, he becomes an unbelievable player (IMO), so there's value in knowing who's a shooting surge from being good and who's a shooting surge away from stardom, and picking the latter. If a team takes a player who needs everything to go right development/shooting wise to be decent over a player who needs everything to go wrong to be no more than decent, even if the former player ends up slightly better, it's not a good pick. It's a bad pick that they got severely lucky on OR they are so good at developing players that the opportunity cost is not the second player they passed on being decent, but the second player having "everything go right" and becoming a way bigger prize than the guy they got. Or to put it another way, you want the player who turns from regular sized Mario into Fireball Mario when he gets a mushroom, not from little Mario to regular sized Mario (and the player that turns from regular sized Mario to little Mario when he gets hit, instead of turning from little Mario to dead when he does)


Top
 Profile  
 
PostPosted: Sun Feb 09, 2014 4:43 pm 
Offline

Joined: Mon May 20, 2013 11:18 am
Posts: 51
Good call. I was wrong to paint Lillard as a case of the "surprise shooter". Nearly 40% from range on 7 attempts and 88% FT is elite.

re: Parsons and Lowry
Parsons is an excellent example. He fooled the model completely (1.4 EWP). Terrible FT shooter, mediocre 3pnt shooter.. not to mention his block and steal rates. The model liked Lowry quite a bit (6.4), but mostly because of the steals. I agree that the shooter was difficult to predict and largely account for his surprising level of impact.

On the general point... I do think one of the better drafting heuristics is to look for guys who are a good shot away from being great players. Andre Roberson was my guy who fit that mold in the last draft. All he needs is a corner three, which there is reason to believe he can do, and you have a starting 3. Dipo is another one who makes sense. Noah Vonleh is the guy I would highlight for that idea in the draft.


Top
 Profile  
 
PostPosted: Sun Feb 09, 2014 4:51 pm 
Offline

Joined: Wed Mar 13, 2013 7:16 am
Posts: 18
VJL wrote:
Historically, the EWP catches quite a few notorious busts: Morrison (2.7), Olowakandi (4.3), Flynn (3.9), Wes Johnson (3.1), Mayo (3.2), Joe Alexander (1.7), Fizer (3.7).

The clearest case going the other direction, where the EWP really didn't like a high pick and that guy turned out to be a star is Deron Williams (3.4). There are some other fringe cases like Brandon Roy and Russell Westbrook, but at least those guys were predicted to be starters (5+) . This is bad news for Andrew Wiggins who at 4.9 is admittedly higher than the busts I just mentioned, but flirting with dangerous territory.

Is it safe to assume that Wiggins' score may be affected by his average steal rate? Which is affected by his coach/team system?


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 95 posts ]  Go to page 1, 2, 3, 4, 5 ... 7  Next

All times are UTC


Who is online

Users browsing this forum: J.E. and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group