Crow wrote:So in this point sample, the models and mocks vastly outperformed the actual draft on future vorp. Blending mock with models helped a modest amount and a modest improvement is still worth going after.
More testing data needed but I'd say teams would be wise to build more and better models and employ more folks to do it. If they don't go after individuals for exclusive deals, it might be worth it for a group to come together and market a set of models, unblended and blended, in future years on a nonexclusive or exclusive basis whichever pays / rewards better.
The first part isn't a surprise - the little historical testing I've done has my model outperforming the actual draft by quite a bit. This is without any scouting (ie subjective) data and combine results. This is why I've decided I will test the last 17 seasons - to fully flush this out. Takes a while. Pretty much every "surprise" quality late pick (in retrospect) my model would have tabbed as being a much better prospect than their actual draft position would suggest (Boozer, DeAndre Jordan, Draymond Green, Milsap, Kevin Martin, Arenas, Chalmers, etc). Almost every horrible bust my model didn't like nearly as well as their actual draft position - although Michael Beasley does stand out as a big whiff in my model, although I think any model ever created would have whiffed there.
There isn't a single true NBA superstar from college that my model wouldn't have tabbed super high. Tim Duncan has the highest single season college rating ever (since '97) by a longshot. Anthony Davis is the highest rated college 18 year old who went pro - followed by Irving (very limited Duke data), Durant, a little drop to Carmelo, another drop to Bosh, another drop to Favors, then we hit the bust Eddie Griffin. Griffin's college rating was 153 compared to AD's 197, Irving's 193, & Durant's 184 - he would not have projected anywhere close to those guys - BUT he would have projected very well in general (if he didn't have any bad limiters like high personal foul rate, etc - which is why I need to fully apply the model to all the past seasons).
Highest rated guards (college rating) 21 years old or under in college who played in the NBA the next season: Steph Curry & Allen Iverson both had 203 ratings as 20 year olds, followed by Kemba Walker, Dwyane Wade, Kyrie Irving, & Ray Allen. Since my model uses the college rating (broken down at every subset) as a base for the NBA career projections, suffice to say these guys would all have been considered certainties for having a good/great NBA career.
I'll take back my model not missing any NBA superstar - Westbrook had a 140 rating as a 19 year old in college, I'm guessing my model would have pegged him somewhere in the 5th to 10th best prospect in his draft - that's probably not being tabbed "super high". Not quite the Irving, Harden, Chris Paul, Mike Bibby, Derrick Rose, John Wall, Mike Conley, TJ Ford, John Wall, Tyreke Evans, & Rondo. Those are the highest rated 18 or 19 year olds PGs/combos who played in the NBA the next season. The next in that list was William Avery, who would be a miss, but he's a ways down from the top guys. Avery (in the '99 draft) rated better than Westbrook in college. There is NO doubt I would have DEMANDED Kevin Love be drafted instead of Westbrook in his draft - Beasley or Love almost certainly would have been my top prospects. I also would have lauded the Timberwolves for taking William Avery 14th in 1999. I would have been wrong.
Looking closer at Avery, he was bad as a frosh, great as a soph. I use weighted two seasons from college, he would not have been rated higher than Westbrook by the model. I'm just glancing over my spreadsheet of draftees that has their last college season rating.
My model doesn't "find" every guy by any means - Eric Bledsoe was not liked by my model. But, I am about certain that any player rated very highly by my model who drops to the 2nd round or goes undrafted is a much better prospect than the scouts give them credit for - and almost any guy my model HATES who ends up going lottery will be a bust. The exceptions from these outliers are few and far between from what I've seen so far. I plan on making all my historical results public before the draft for complete scrutiny.
All that being said - I am certain a quality model COMBINED with quality scouting & taking into account some pertinent combine stuff (w/o going overboard) would do much better than what teams have generally done historically. Much lesser chance of the horrible bust. Much better chance at getting that quality 2nd round or undrafted guy who very well could pan out to be a nice contributor for years. A quality model probably would probably really help D league teams draft. A quality model would help WNBA teams draft.
So Crow, your second point - I completely agree. There is so little risk for NBA teams to hire a team of guys whose whole responsibility is to combine their skills to best model NBA career projections (MAN I wish I could build databases & data mine worth a crap - but others can, no I in TEAM) - coming out of college, from foreign leagues, from D Leagues, etc. One could incorporate the high level AAU leagues now that they have greatly improved their statkeeping (making it easier to adjust for pace, SoS, etc). One could model from all the past Summer League & Exhibition seasons to maybe help tab current players who might already be ahead or behind projections during their summer league & exhibition. Projections could be updated daily. Projections would make salary & free agent decisions easier & better planned out long term. The work could be extended to D League & WNBA, PLENTY of work for the "modelers" to earn their paychecks - which combined might not reach NBA minimum salary for one player (depending on how big the model "team" is).