Thoughts on improving models

[Crow]

I had some thoughts towards trying to make more sophisticated, sensitive, integrated analytic models today. It was surface hopscotch set off by trying to remember or find something new / helpful in a "other sport" adjusted plus minus model of team and lineup performance as referenced in the recent thread on newly reported adjusted plus minus values. (MSBuck, I am thinking hockey or soccer, in last 4-5 years? Haven't found it yet, but in a brief look I didn't expect to. Yet.)

Without further ado, here are the fragmentary initial notes, fwiw:

Re-read Piette's Sloan paper on social networks. Successful plays are often the result of several important contributions from different players. Maybe connections, pathways are not minor interactive terms but a major or the major thing to study, instead of individuals, posited as discrete entities.

Look closer at paper using random set theory and the four factors to analyze "formula" for championship success. Simple correlation of these factors and winning championships has been done here, but what wins championships is not any one factor but a set of factors together, interacting. Take this concept and apply to individual plays below.

RPM uses a box score prior to improve model performance. We know that most box score metrics including BPM either leave shot defense or estimate it off team performance (and without player on/off court sensitivity, much less actual role in the plays). Does this create an offensive bias? Could a better prior be tested, found and substituted? People have previously poo-pooed 82games counterpart data as too insensitive and simple blends of counterpart and team shot defense as subjective and not compelling. But what about using video based player tracking data in the prior? Instead of just blending player tracking data findings with RAPM as Andrew Johnson has done, integrate them into one model. Make a true player tracking influenced plus minus model. And / or use a shot defense credit system based on a certain % for counterpart performance from video verified matchups at start of play, end of play and a certain amount for shared defensive responsibility. Maybe use machine learning to optimize the weights for fitting the plus minus regression model to the data.

Or go beyond output data as priors to look at inputs from video. Raptors apparently "score" their players defensive movement vs. "a ghost system" of coach identified ideal movement. One could go beyond movement to score actions. Either prior identify 5-20-50-whatever common play actions (offense and defense), identify them in plays and score their importance or maybe take say 100 to 1000 games of SportVu video / data and have 3 different people score the action. Everything or simplified to the 3-5 most important actions in every play (point-shares?) compile and cluster the data and then fine-tune the language to be used in future to increase simplicity and consistency. Discovery of these categories instead of prior assumption. This was prompted by brushing past another sport model that focused on shots instead of points, because shots in that sport predicted impact better than just points. If you think adjusted plus minus is a black box that does explain "how", give it "how" from traditional analysis (eyes), aided by technology. My understanding is that a certain analytic entity believed to have found "value-added" from comprehensive coding of play defense as simply "man" or "zone". If you coded plays 10-100 ways that would be a valuable resource directly and as a input data based prior for plus minus models. probably in addition to boxscore outputs but try separate and together.

Or what about using adjusted four factors (or raw player and team while on court four factors data) as priors instead of BPM? And using machine learning to refine the prior input and the model performance?

Writing this down, helps me some to remember my thoughts and perhaps pursue / refine at least some of them later. Maybe something in it helps someone else? There is that chance.

[Crow]

P.S. J.E., have you seen this by J Rao & colleague? http://scholar.google.com/scholar_url?u ... ws=480x245

or if that doesn't show, it is the first link here https://scholar.google.com/scholar?q=ju ... _sdt=0%2C5

Did it direct or influence your game state adjustment to RPM? If not, are they roughly comparable in findings / impact?