Predictions 2013-14

[Mike G]

At .40 of the season, we've cut the error to 8.80 on average.

Code: Select all

Bobb   7.75      ATC    8.85
jBro   7.83      deep   9.01
jank   8.53      Yoop   9.25
ncs    8.59       eW    9.46
hDon   8.59      13Py   9.89
416x   8.80      fpli  10.18

[v-zero]

Are you aggregating the errors to come up with the combined error number for the entrants, or are you aggregating the predictions, and then calculating the error? The latter, in my opinion, is the more interesting value.

[Mike G]

Here are our avg/aggregate predictions for teams, and pythagorean projections; sorted and ranked by the over/under (Diff)

Code: Select all

. overachievers                 underachievers
diff   tm    avg   pyth+      diff    tm    avg  pyth+
27.5   Phx    23    50        -0.2    LAL    33    32
20.2   Por    39    59        -2.2    Sac    31    29
13.8   Ind    52    65        -3.5    Hou    54    50
7.7    Okl    54    62        -3.9    LAC    57    53
7.3    Phl    19    26        -4.8    Den    43    38
7.0    Bos    27    34        -5.3    Det    40    35
6.8    GSW    46    52        -6.0    Uta    29    23
6.1    Cha    28    34       -12.1    Mil    32    20
5.8    Tor    39    45       -12.7    Cle    39    26
5.6    SAS    56    61       -15.1    Chi    53    38
4.0    Min    43    47       -15.6    Mem    51    35
3.0    Atl    41    44       -18.2    NYK    46    28
2.9    Orl    25    28       -25.7    Brk    54    29
2.4    Dal    44    46                    
1.3    NOP    38    39                    
0.8    Mia    59    60                    
0.7    Was    37    38                    

The average absolute difference/error is 8.30
Pyth+ is just current wins plus (remaining games * Pyth%) -- no SOS, either to date or in future schedules, is considered.
When I replace pyth+ with the simulation-generated W on this page -- http://www.basketball-reference.com/fri ... f_prob.cgi -- we get an aggregate error of just 7.65
Without SOS, I've projected the Pacers to win 65. With that input, b-r.com says 61 wins.

[AcrossTheCourt]

So you're doing average absolute error and not RMSE?

I thought the Arturo Galetti/Wins Produced predictions would be interesting to add or at least compare:
http://www.boxscoregeeks.com/teams

I'm getting an average absolute error of 10.6 for those predictions (and a 13.1 RMSE.) They really whiffed on the Blazers (28.6 wins) and Pacers (39.9 wins ??).

[Mike G]

Here are those avg errors relative to b-r.com's predicted final wins; and RMSE relative to same

Code: Select all

average absolute error                  root of mean squared error
jBro   7.15      416x   8.04          hDon   9.66      ATC    10.41
Bobb   7.21      jank   8.08          jBro   9.72      jank   10.87
ncs    7.88      Yoop   8.63          Bobb   9.88      Yoop   11.04
hDon   7.92       eW    8.79          416x   9.95      13Py   11.36
deep   7.93      13Py   9.15          ncs   10.04       eW    11.65
ATC    8.02      fpli   9.39          deep  10.39      fpli   11.82

The biggest clunker among us is no longer my picking Brk to win 60 (b-r sez 31)
This honor is now nearly shared by fplii and jfrank giving Phx 18 wins -- about 30 short of their pace.

I'm not sure that squaring (exaggerating) the errors gives us a better picture. Some of these are just flukish and crazy. It might be better to take the average square root of errors. Kind of like collapsing runaway scores.

Code: Select all

avg square root of errors
jBro   2.4140    416x   2.60
Bobb   2.4145    hDon   2.63
deep   2.48       eW    2.67
jank   2.57      Yoop   2.68
ATC    2.57      13Py   2.78
ncs    2.59      fpli   2.87

[nbacouchside]

So you're doing average absolute error and not RMSE?

I thought the Arturo Galetti/Wins Produced predictions would be interesting to add or at least compare:
http://www.boxscoregeeks.com/teams

I'm getting an average absolute error of 10.6 for those predictions (and a 13.1 RMSE.) They really whiffed on the Blazers (28.6 wins) and Pacers (39.9 wins ??).

That makes sense to me as WP is a pretty rotten metric for prediction. It's good as a way of describing what happened, but it always stinks compared to other metrics at predicting the future.

[jbrocato23]

That makes sense to me as WP is a pretty rotten metric for prediction. It's good as a way of describing what happened, but it always stinks compared to other metrics at predicting the future.

Why, because it adds up to the point margin? You could make literally anything add to the point margin with a "team adjustment," which mystic rather hilariously showed once at the realgm forums. I'd argue that wp is as useless as an explanatory metric as it is as a predictive one. Regardless, I think the real value of Berri and wp is that it has got a lot of people interested in and excited about advanced nba stats.

[Mike G]

Teams have suddenly been playing more like they're supposed to, and our avg error is now just 7.41

Code: Select all

Bobb   6.22      416x   7.74
jBro   6.44      jank   7.76
hDon   6.86       eW    7.89
ncs    6.88      Yoop   7.98
ATC    7.27      fpli   8.87
deep   7.58      13Py   8.90

Again, this is relative to b-r.com's forecast.
http://www.basketball-reference.com/fri ... f_prob.cgi

[nbacouchside]

That makes sense to me as WP is a pretty rotten metric for prediction. It's good as a way of describing what happened, but it always stinks compared to other metrics at predicting the future.

Why, because it adds up to the point margin? You could make literally anything add to the point margin with a "team adjustment," which mystic rather hilariously showed once at the realgm forums. I'd argue that wp is as useless as an explanatory metric as it is as a predictive one. Regardless, I think the real value of Berri and wp is that it has got a lot of people interested in and excited about advanced nba stats.

Yeah, that's true. Good is relative here. It's good in the sense that it's a bit better at explaining the past than it is at predicting future events, for which it is basically useless. But yeah lots of things could be just as predictive or even more so with the kind of team adjustment they throw in.

[Bobbofitos]

Teams have suddenly been playing more like they're supposed to, and our avg error is now just 7.41

Code: Select all

Bobb   6.22      416x   7.74
jBro   6.44      jank   7.76
hDon   6.86       eW    7.89
ncs    6.88      Yoop   7.98
ATC    7.27      fpli   8.87
deep   7.58      13Py   8.90

Again, this is relative to b-r.com's forecast.
http://www.basketball-reference.com/fri ... f_prob.cgi

Phew, glad I'm back in front. Was stressed for a while!

[Mike G]

The T-Wolves are 18-20, and b-r.com's simulation machine expects them to go 26-18 -- same as the Blazers -- the rest of the way.
They're still just 46% likely to make the playoffs. They'll have to pass (most likely) either Phx or Dal, both of whom have a 2/3 chance of making the top 8; but less chance than Min of reaching the Finals.

Wolves are 0-11 in games decided by <5 points. Is that just as unlikely as their going 11-0 ?
Move 11 games from the L to the W column, and they're 29-9 -- again, tied with Por.

The top 7 teams in the league now (by SRS) are the Pacers and 6 Western teams. Min (4.84) is better than Mia (4.24), Hou (4.07), Phx (2.68), Tor (2.50), Dal (2.43), and 5 other playoff-bound teams from the East.

[Crow]

T-Wolves weakest Factors are the biggest- own and opponent eFG% (25th and 30th respectively). This reflects major personnel, strategy and / or effort issues. It is at least in part a strategy imbalance issue (I think) as I see they are top 8 in the other 6 Factors and top 3 in four of them. Over the last 4 seasons, Adelman's teams 15th or worse on opponent eFG% and average a ranking of 22nd. on own FG% they averaged a 21st rank but only because of a 10th place in Houston in 2009-10. These two categories are by far his team's worst Factors so it is not just a partial season fluke. They have averaged 6th fastest on pace and 5th lightest on fouls given. These are probable significant parts of the strategy problem. But one can also think about talent level and type of talent and whether the strategy was the right one for that set of players to get the most of out of them.

Their expected win - loss record is 24 -14 and yet they are 18-20. I don't recall seeing a greater imbalance especially for just half a season. How much is "not being clutch" vs. unlucky? The only other teams to be more than 2 wins different than expected are Toronto at -3 and Philly at +3. Adelman has not exceeded expected wins since 2009-10 and averages -3.75 over the last 4 seasons (likely the worst out there or nearly so). I know behavior in blowouts can affect this ratio but this is hard to talk down. 0-5 this season in games decided by 3 points or less.

[AcrossTheCourt]

I think what's helped us a lot are the New York teams slowly climbing to 0.500 ball.

By the way, Arturo Galetti/Wins Produced prediction is at 8.4 (absolute error) and 10.6 RMSE. It would be nice to see them discuss why WP's usually underperforms compared to other predictions/metrics.


Now that a large portion of the season is gone and they're no longer a fluke, what did we miss with the Phoenix Suns? How could a prediction methodology change to incorporate the 2014 Suns?

[Crow]

Instead of building team projections pretty much entirely on simple summation of player level data, it might be possible to construct a typology of team types (based on the set of their current players, projected offensive and defensive team efficiencies, redundancies and gaps in boxscore stats and the Four Factors and maybe the coach and perhaps where the team has been recently record-wise) and look at the player level data and the data for the relative degree of success for those types of teams (outright and perhaps in a year to year progression, a team type aging curve). Maybe there are some typical player interaction effects that could be uncovered thru the modeling that would improve the projections. I am not sure if any of the present models are dynamic but it might help to try to make them dynamic or more dynamic.

I am not saying it would have been very likely to produce a model that would have specifically predicted greater success for the Suns this year than the consensus but it might help some on average or lead to some other useful thinking and discoveries. The Suns are both the #2 team on own 3 pt attempt rate and #4 at limiting 3 pt attempts by opponents. Was there an adequate basis for expecting this strategy or these outcomes? How do teams with both of these achievements tend to do? What are the typical spinoff Factor level effects? What is the "adjusted" overall value of these strategies or achievements?

I know the above is rough and speculative but that was my reaction to the question.

[Bobbofitos]

I don't think the Wolves' under performance in close games is randomness at all, having said that, they've been obviously fairly unlucky to underrate their Pyth by this large a margin.