2012-2013 predictions

[Neil Paine]

Updated:

Code: Select all

rk      naive pace      RMSE            rk      .500 prior      RMSE            rk      pyth prior      RMSE
----------------------------            ----------------------------            ----------------------------
1       Bovada          5.98            1       Bovada          5.79            1       Bovada          5.66
2       Hollinger       6.37            2       Hollinger       6.23            2       Hollinger       6.17
3       ESPN Summer     6.55            3       ESPN Summer     6.34            3       ESPN Summer     6.24
4       Schaller        6.78            4       Schaller        6.56            4       Schaller        6.49
5       LV Hilton       6.83            5       LV Hilton       6.63            5       LV Hilton       6.49
6       np "dumb"       7.15            6       np "dumb"       6.82            6       np "dumb"       6.66
7       Bobbofitos      7.41            7       Bobbofitos      7.15            7       Bobbofitos      7.02
8       v-zero          7.43            8       DSMok1          7.17            8       DSMok1_2        7.04
9       DSMok1_2        7.47            9       DSMok1_2        7.17            9       DSMok1          7.04
10      DSMok1          7.47            10      v-zero          7.22            10      v-zero          7.04
11      WoW1            7.68            11      WoW1            7.42            11      WoW1            7.30
12      WoW2            7.97            12      WoW2            7.73            12      WoW2            7.61
13      TeamRankings    8.12            13      TeamRankings    7.77            13      TeamRankings    7.63
14      2012reg         8.44            14      SCHOENE         8.23            14      2012reg         7.99
15      SCHOENE         8.47            15      2012reg         8.23            15      SCHOENE         8.11
16      all41          12.34            16      all41          11.93            16      all41          11.89
----------------------------            ----------------------------            ----------------------------

Huge surges by Hollinger & Schaller! (Schaller won last year as well -- perhaps onto something?) Vegas & ESPN Summer continue to do well, as is usually the case. Meanwhile, SCHOENE still struggling with a regressed version of last year's standings.

[wilq]

Here are all of those in a table (I scaled everybody to 1230 total wins):

First of all, thanks for preparing that table but out of curiosity, why did you scale them?
Isn't it altering the predictions? Especially when Vegas for years has done "uneven" ones.

(Schaller won last year as well -- perhaps onto something?)

He also had a good season in 2010-11... so probably yes.

[Bobbofitos]

Update time!

[Neil Paine]

I was just going to wait until the end of the season, but what the hey, why not give an update going into the last week...

I'm not going to jump thru all of the Bayesian hoops I've been doing all season, just because all of those estimated records are pretty much going to be interchangeable with naive pace at this point. So here's how everybody stands right now:

Code: Select all

Projection       RMSE
---------------------
Hollinger       5.949
Bovada          6.125
ESPN Summer     6.587
Schaller        6.641
LV Hilton       7.010
v-zero          7.259
Bobbofitos      7.276
np "dumb"       7.397
DSMok1_2        7.463
WoW1            7.510
DSMok1          7.511
WoW2            7.809
TeamRankings    8.264
SCHOENE         8.474
2012reg         8.986
all41          12.518

As for why I scaled everyone to an average record of 41-41 -- I think it's silly to dock people for rounding errors or if the totals don't add up exactly to 1,230 wins. It takes literally no predictive skill to guess that there will be 1,230 games won in the upcoming season, so why not put everyone there as a starting point?

[Neil Paine]

I also wanted to point to a couple of Phil Birnbaum posts from last week that I thought were relevant in this thread:

http://blog.philbirnbaum.com/2013/04/ac ... light.html
http://blog.philbirnbaum.com/2013/04/ac ... ght_4.html

It's important to stress that, in a single trial like this, the results of our prediction contest themselves are more the product of luck than skill. And I'm not just saying that because my own predictions finished in the middle of the pack (I named them "dumb" and openly wrote that most systems should beat them -- it's not like I staked my reputation on them or anything). I just want to remind everyone that the true differences in picking teams are not as wide as the distribution of RMSEs would have us think. Hollinger, Schaller, & Vegas are probably truly above the mean, especially since they've been doing well in this contest for several years, but beyond that be careful about any conclusions you draw.

[v-zero]

I agree, Neil, it is a case of very many weighted coin tosses, but I think there are cases where (e.g. Houston) most people were dead wrong about their chances, but where others have made an accurate prediction. Analysis of the microcosm like this can be both interesting and revealing. It surprised me that anybody had Houston below 41 wins with a decent PG, and young SG destined to be the best the league has seen since Jordan (maybe), and an underrated, underused ex-Bull who knows all about playing a role, and great D.

Back on the note originally touched though: Consistent season-to-season results will gradually build up a Bayesian view of a pundit which says whether they have genuine skill beyond luck.

[Crow]

WOW versions in 10th and 12th place out of 16 entries, with the ones lower very crude / "dumb" or just not wise.

Given its weak to vey weak performance almost every year of these contests, Schoene seems either to need a different / more sophisticated comp player stat weighting system, an entirely new approach (if it is be used to make team predictions) or something else.

[John Hollinger]

Fawning articles need to be written about my dramatic comeback in the predictions standings. I was in dead last on Jan. 1.

All the credit has to go to them. My predictions showed a lot of willpower and toughness being able to come back from so much adversity. A lot of other predictions would have just packed up and gone home. But this was a group of predictions that never gave up, that played for each other, and just kept plugging away. Nobody deserved it more than these predictions.

[Bobbofitos]

Fawning articles need to be written about my dramatic comeback in the predictions standings. I was in dead last on Jan. 1.

All the credit has to go to them. My predictions showed a lot of willpower and toughness being able to come back from so much adversity. A lot of other predictions would have just packed up and gone home. But this was a group of predictions that never gave up, that played for each other, and just kept plugging away. Nobody deserved it more than these predictions.

They don't hand out rings for winning the reg season predictions, they have to hold up in the playoffs!

[Crow]

Not sure if scoring system is exactly the same but it appears that every entrant in the contest last season was better than the best this season.

viewtopic.php?f=2&t=7830&hilit=PREDICTIONS&start=195

My winning predictions last season were about 40% better. Even the worst prediction last season was 10% better than this season's winner. Kinda weird. I would have thought last season would have been harder to predict given the lockout.

[v-zero]

It looks like last year the absolute error was used, which has a smaller spread than the RMSE being used this year (and is actually a natural loss function for estimates for the median, not the mean). Also, consider that last year's season was only 66 games, and that RMSE ~ SQRT( NUMBER OF GAMES).

[Crow]

Alright, thanks for the reply. I realized the 66 games factor shortly after I left my computer. I guess in a shorter season there might be relatively more random variance from true talent and what people predict on that basis though? But probably the difference in performance is not as big all things considered.

[schtevie]

Any interest on the part of participants in a retrodiction contest, based on actual minutes played?

[v-zero]

Alright, thanks for the reply. I realized the 66 games factor shortly after I left my computer. I guess in a shorter season their might be relatively more random variance from true talent and what people predict on that basis though? But probably the difference in performance is not as big all things considered.

There is likely little difference in errors between last year and this year. Like I said we have VAR ~ N, that's the whole shebang.

[v-zero]

Neil I'd be interested in the error on the average of the APBR posters (those that post frequently/consistently), could you compile that?

Also this is now deeply ironic:

As for why I scaled everyone to an average record of 41-41 -- I think it's silly to dock people for rounding errors or if the totals don't add up exactly to 1,230 wins. It takes literally no predictive skill to guess that there will be 1,230 games won in the upcoming season, so why not put everyone there as a starting point?