2015-16 Team win projections

[bbstats]

Any update?

[Mike G]

Relative to b-r.com simulations, avg errors are

Code: Select all

AJ   4.71      bbs  5.29      yoop 6.17
km   4.74      Crow 5.32      itca 6.30
DF   4.84      rsm  5.44      nr   6.38
KF   4.86      fpli 5.83      EZ   6.74
Cal  4.88      MG   5.88      DrP  6.80
tzu  5.04      snd  6.02      Dan  7.18
DSM  5.05      BD   6.15      taco 7.45

And RMSE:

Code: Select all

km   5.67      tzu   6.52      itca  7.43
AJ   5.71      rsm   6.60      BD    7.70
Cal  5.72      Crow  6.73      nr    7.83
DF   5.92      fpli  6.85      EZ    8.32
KF   6.09      snd   7.22      taco  8.48
bbs  6.17      MG    7.27      Dan   8.59
DSM  6.45      yoop  7.33      DrP   8.60

[EvanZ]

I agree. That's why I said although there's a hint, probably we didn't get worse in reality.

But I believe this confirms our ability to predict has not been improved meaningfully at all, if it's improved.

Can we at least say the RAPM-based methods are better than the non-RAPM based ones?

[EvanZ]

Moreover, if we all agreed on the minutes projections (which is how I think these projection contests should operate, because we're actually more interested in the player valuation models than the minutes projections), which method would win? And if I just use something "simple" like 2-year RAPM, how far behind the winner would I be?

One more thing. Do the PyWin projections take into account games already won? Seems like with the Warriors at 42-4 they should be on pace to win more than 66 games if we just look at the PyWin% for the rest of the season and add that to the accumulated wins from the first 46 games.

[Mike G]

. Do the PyWin projections take into account games already won? Seems like with the Warriors at 42-4 they should be on pace to win more than 66 games if we just look at the PyWin% for the rest of the season and add that to the accumulated wins from the first 46 games.

Yes, at least b-r.com does. They do 7500 simulations of the remainder of the season, average the resulting wins and losses.

Code: Select all

tm    W     L    Current  Remain  Best   Worst
GSW  69.3  12.7   42-4     27-9   76-6   60-22

Their remaining 36 games ranged from 34-2 to 18-18 in this morning's iteration.
27.3-8.7 would be the avg

It's good that the Spurs are challenging for the #1 seed. The Dubs still have to try and win games. Otherwise, they might just coast in.

Meanwhile, I just retrodicted some stats from last year onto actual minutes played this year, and I get these average errors, wins projected to 82 games:

Code: Select all

ws/48  5.3
BPM    5.8
RPM    6.1
eW/48  6.5
PER    6.9

PER may be off, as it seems heavily regressed -- predicting 30 wins for the Sixers, 50 for the Spurs.

It's most strange. Here are correlations between these stats (from last year) and this year's mpg for 381 players who played both years:

Code: Select all

stat   2015  2016
BPM    .61    .54
e484   .54    .49
RPM    .51    .49
PER    .49    .44
WS/48  .34    .32

I used actual (2016) rookie rates to get team rates/minutes; including Julius Randle (rather than his terrible 15 minutes last year). Also Paul George 2014 rather than 2015.

[AJbaskets]

Moreover, if we all agreed on the minutes projections (which is how I think these projection contests should operate, because we're actually more interested in the player valuation models than the minutes projections), which method would win? And if I just use something "simple" like 2-year RAPM, how far behind the winner would I be?

One more thing. Do the PyWin projections take into account games already won? Seems like with the Warriors at 42-4 they should be on pace to win more than 66 games if we just look at the PyWin% for the rest of the season and add that to the accumulated wins from the first 46 games.

I am not sure that actually solves the problem, for example, if PT-PM thought Durant was not going to be good last year and used the consensus minutes projections I would have beaten the field on OKC's wins estimates even though that player estimate was wrong.

I *think* combining a box score estimate and RAPM, adding an age curve, mean regression and team change regression all add incrementally, not sure exactly how much.

[EvanZ]

I am not sure that actually solves the problem, for example, if PT-PM thought Durant was not going to be good last year and used the consensus minutes projections I would have beaten the field on OKC's wins estimates even though that player estimate was wrong.

True, then I would suggest each participant simply give a list of player ratings (per 100 possessions) and wins could be calculated based on actual possessions played during the season. Wouldn't that work? I mean, aside from the fact that it's not a "Win Projection" contest anymore.

[Mike G]

2015 rates applied to 2016 minutes.
Projected is what b-r.com said yesterday. Errors on the right.
Scaled to 41 wins avg. No aging curve.

Code: Select all

eW    WS   BPM   RPM       Proj   err:  eW   WS  BPM  RPM
55    52    52    55        46    Atl    9    6    6   10
40    43    47    49        47    Bos    7    4    0    2
30    30    26    21        25    Brk    5    6    2    4
26    36    39    35        42    Cha   15    6    2    6
46    47    41    45        45    Chi    1    2    4    0

48    48    51    63        56    Cle    8    7    5    8
41    45    44    48        43    Dal    2    2    2    5
17    21    26    19        32    Den   16   12    6   13
39    40    33    39        45    Det    6    5   12    6
58    61    67    69        69    GSW   12    9    2    0

57    51    49    55        40    Hou   17   11    8   15
53    46    49    48        45    Ind    8    1    4    4
57    51    56    50        51    LAC    6    0    5    1
36    25    20    22        19    LAL   18    6    2    4
43    48    57    55        43    Mem    1    5   14   12

eW    WS   BPM   RPM       Proj   err:  eW   WS  BPM  RPM
53    40    34    33        43    Mia   10    2    9    9
36    33    31    32        35    Mil    2    2    4    3
35    28    28    21        28    Min    7    0    0    6
44    45    47    45        34    NOP   11   11   14   11
26    27    23    23        38    NYK   12   12   15   15

63    55    63    57        56    Okl    7    1    8    1
25    27    22    21        37    Orl   11    9   15   16
14    24    13    10        17    Phl    3    7    3    6
29    38    36    30        27    Phx    2   10    9    2
28    44    44    41        37    Por    9    7    7    4

49    39    46    42        38    Sac   11    1    8    4
67    56    62    67        66    SAS    1   10    4    1
48    50    47    47        53    Tor    4    2    5    6
38    40    39    39        39    Uta    1    1    0    0
29    40    37    47        38    Was    9    2    1    9

[J.E.]

The way Joe Sill used to run this, wich is also my preferred way, is ignore possessions that have rookies in them, since we have no estimates for them and some of them (obviously) perform better than others

Also, running things with an aging curve would produce more accurate results - although the question becomes whether we use different aging curves for the different metrics. I'm not even sure whether they exist for every metric

Further, some of these projections need to be regressed to the mean more than others (e.g. due to 'effect of leading' being present in RPM, etc)

[Mike G]

I wonder how the aging curve varies along a spectrum from "young player on older team" to "team of young players".
Regressing to the mean, even knowing the minutes used? I did try some partial regressions, without much improvement.

Including current rookie minutes and rates, a similar effect is laid on all metrics. That shouldn't affect how the metrics predict, should it?

[J.E.]

Including current rookie minutes and rates, a similar effect is laid on all metrics. That shouldn't affect how the metrics predict, should it?

If you're trying to do a true out of sample test, you simply can't use rookie rates from this season

Regressing to the mean, even knowing the minutes used?

RPM expects players to perform worse when up big, and vice versa. E.g. a team that performs as a +15 when tied will perform significantly worse when actually up 15. Any team win projections will have to account for that, or your "RPM projections" will turn out to be wider than they would have been if you simulated possession-by-possession, adjusting for current lead.

Aside from that, I'm sure all metrics benefit from "regressing to the mean" when projecting an entire season, and it might vary (slightly) from metric to metric how much regression to the mean is optimal

[permaximum]

RPM expects many things which make it automatically worse for outliers such as the best players and the worst players yet most here use it for those "outliers". Ironic.

[Mike G]

Squared and not, there are clear 1st and 2nd divisions.

Code: Select all

avg error    avg error      sqd error    sqd error
KF   4.72    BD   6.08      Cal  5.81    yoop 7.45
km   4.78    MG   6.10      AJ   5.82    MG   7.48
DF   4.85    snd  6.10      km   5.90    snd  7.49
AJ   4.86    itca 6.30      DF   6.05    itca 7.65
Cal  5.04    DrP  6.53      bbs  6.34    BD   7.82
tzu  5.05    nr   6.54      KF   6.38    nr   7.92
DSM  5.12    yoop 6.60      DSM  6.59    EZ   8.39
bbs  5.16    EZ   6.86      tzu  6.61    DrP  8.54
rsm  5.56    Dan  7.21      rsm  6.63    Dan  8.55
Crow 5.58    taco 7.34      fpli 6.96    taco 8.58
fpli 5.84    15py 8.82      Crow 6.98    15py 9.86

[Mike G]

Setting the best guess (for each team) to zero, everyone's current (absolute) difference from that best guess:

Code: Select all

tm    KF   km   tzu  AJ   DF   bbs  DSM  Cal  rsm Crow  Avg  fpli snd  MG   BD  itca  DrP  nr  yoop  Dan  EZ  tac
Atl    0    2    0    5    3    2    5    6    5    1    4    2    7    5    4    5    9    3    5    3    8    6
Bos    0    1    0    0    0    3    1    2    1    2    3    0    3   10   12    3    7    2    5    7    3    5
Brk    0    4    3    1    0    3    0    2    3    0    3    5    3    5    5    4    4    4    4    7    7    3
Cha    4    3    4    3    3    3    4    5    9    2    6    6    7    6    6   16    6   15    0    3    7   14
Chi    0    0    3    4    3    2    3    1    6    2    4    5    4    0    3    8    3    5    6    5    7    8

Cle    0    0    1    0    2    0    1    1    2    0    2    0    3    6    1    0    0    1    6    1    3    4
Dal    1    4    1    1    2    2    0    3    2    4    3    2    0    3    0    4    2    2    3    9    0    9
Den    8    9    7    6    7    7    6    7    8    5    6    5    8    6    2    8    4    3    6   11    0    1
Det    4    3    1    4    5    3    5    2    7    7    6    3    5    0   12    3    3   12   12    9   15    8
GSW    7    5    7    0    3    2    5    4    1    5    5    5    7   13    5    7    0    6    6   13    5    7

Hou    6    5    9    4    5    7    6    3    6    8    6    6    7    6    8    7    0    9    4   13    9    6
Ind    5    2    5    3    5    7    8    4    3    3    4    0    5    2    3    1    7    9    8    0    2   15
LAC    2    1    1    1    2    1    1    4    5    2    2    2    2    5    4    1    0    7    0    6    1    2
LAL    4    6   10    3    3    1    2    5    0    3    5    5    2   14    7    5    0    6    6    2    9    8
Mem    1    5    1    6    6    4    3    6    8    5    5    7    1    3    1    3    6    8   11    0    5    7

tm    KF   km   tzu  AJ   DF   bbs  DSM  Cal  rsm Crow  Avg  fpli snd  MG   BD  itca  DrP  nr  yoop  Dan  EZ  tac
Mia    5    4    0    3    4    9    4    4    4    3    5    0    5    8    4    0   10    6    9   11    3    3
Mil    6    5    3    2    2    4    3    4    5    7    5    5    7    7   14    9    0    0    2    5   13   11
Min    1    1    3    2    0    1    3    3    4    1    2    5    4    1    2    2    2    7    7    2    5    0
NOP    5    1    3    5    6    3    7    2    5    6    4    9    6    2    4    4    8    6    4    0    7    1
NYK    8    5    7   11   10    8   12    7   12    8    9    8   13   10   14    4   22    5    6   12   12    0

Okl    1    0    1    2    0    2    0    0    0    3    2    3    3    0    0    2    1    6    1   12    3    0
Orl    2    2    1   10    6    4    3    8    6    9    6    9    3    4    7    6   11    8    5    0   10    9
Phl    9    4   11    8    6    7    8    4    1    6    6    2    9    2    1    9   17    9   10    2    0   12
Phx    9    8    0    5    8    9    7    8    9   10    7    9   11    7    3    8   13   10    1    7    5    8
Por    1    0    3    6    5    2    2    5    5    5    4    7    2    3    4    9    3    1    8    3    2   12

Sac    1    5    2    0    2    0    4    3    2    3    3    4    1    8    3    1    1    6    3    7    6    1
SAS    6    5   10    2    3    5    5    4    0    7    5    4    7    4    7    5    5    2    8    5    6    7
Tor    6    9    7    6    8    6    5    9    6    8    7    9    9    9   10   12    0    2   12    5   10    8
Uta    0    2    1    4    0    0    0    1    2    1    2    5    0    2    0    2   10    1    3    3    1    4
Was    3    6    3    4    3    8    8    4    6    8    5    7    5    0    6    7    4    3    5    6    8    8

avg   3.5  3.5  3.5  3.7  3.7  3.8  4.0  4.0  4.4  4.4  4.6  4.6  4.9  5.0  5.0  5.1  5.2  5.4  5.5  5.6  5.7 6.2

Everyone has at least one best guess (or within 0.5) right now.
Our average prediction is at least 2 wins off for every team.
Tacoman had the Knicks at 10 wins better than the rest of us, on avg. They're doing just that well -- and they fire their coach?

[Mike G]

At All Star weekend:

Code: Select all

km   4.86      snd   6.22
KF   4.89      MG    6.29
tzu  4.96      BD    6.30
AJ   5.05      itca  6.52
bbs  5.13      DrP   6.66
DF   5.13      yoop  6.81
DSM  5.37      nr    6.85
Cal  5.45      Dan   6.91
rsm  5.78      EZ    7.10
Crow 5.79      taco  7.67
fpli 6.15      15py  8.99

Teams have not cooperated of late, and almost all of us are looking worse in the last week.
Exceptions: tarrazu (improved from 5.05 to 4.96), bbstats (5.16 to 5.13), StatmanDan (7.21 to 6.91)
Others had their errors rise as much as .040