Re: 2022-23 team win projection contest
Posted: Wed Mar 22, 2023 8:04 pm
If stretched to all-time I believe my average rank would be higher. I am pretty sure I am the only 3-time winner.
APBRmetrics
Analysis of basketball through objective evidence
https://apbr.org/metrics/
2022-23 team win projection contest
Page 11 of 13
Re: 2022-23 team win projection contest
Posted: Thu Mar 23, 2023 1:09 am
I got 3 wins, 16-17, 18-19, 21-22. Not sure how that corresponds to those percentiles. You can be in rare company with me Crow.
Re: 2022-23 team win projection contest
Posted: Thu Mar 23, 2023 1:42 am
I knew you had at least one but I didn't run down all the totals at this time.
Until your win last season I was sure I was ahead of all and didn't realize til you just spoke up that perhaps last season changed this.
Close between us this season but not at the top.
Until your win last season I was sure I was ahead of all and didn't realize til you just spoke up that perhaps last season changed this.
Close between us this season but not at the top.
Re: 2022-23 team win projection contest
Posted: Thu Mar 23, 2023 2:11 am
Mike G's accounting here has 2 ahead of you in 2017 but we have had several measuring sticks and multiple felt winners at times including in 2017. And who was an official entry and not is another factor.
But not wanting re-litigation of the past.
Tied at 3 with you is fine for me.
Just us as far I know.
Overall, more than a handful of winners and lots of good to great performances.
But not wanting re-litigation of the past.
Tied at 3 with you is fine for me.
Just us as far I know.
Overall, more than a handful of winners and lots of good to great performances.
Re: 2022-23 team win projection contest
Posted: Thu Mar 23, 2023 3:57 am
Yeah, it’s really unfortunate that @caliban didn’t participate this year.nbacouchside wrote: ↑Wed Mar 22, 2023 7:28 pmCaliban the GOAT!Mike G wrote: ↑Wed Mar 22, 2023 6:56 pm Luck is big.
Here's a decidedly incomplete table of entries who were entered at least twice in the last 7 years. The numbers are a sort of 'percentile', as the % of entries that finished beneath you. Middle of the pack is around 50.Caliban did not show up this year, but they were in or near top 1/3 for 6 years straight.Code: Select all
year 2023 2022 2021 2020 2019 2018 2017 # 19 13 16 17 18 20 23 yrs avg Crow 58 46 56 94 28 15 57 7 51 vegas 21 62 88 18 39 50 70 7 49 538 32 77 19 24 22 90 0 7 38 cali 69 63 88 89 65 87 6 77 trzu 53 92 38 47 94 78 6 67 shad 85 81 41 50 40 30 6 55 emin 47 38 13 65 72 95 6 55 KPel 89 31 35 78 4 58 RyRi 59 44 60 52 4 54 sndi 53 33 85 39 4 53 dtka 95 54 50 12 4 53 ncs 63 29 0 65 4 39 eWin 8 0 45 35 4 22 year 2023 2022 2021 2020 2019 2018 2017 bbst 69 71 67 3 69 kmed 17 80 96 3 64 gold 76 56 25 3 52 lisp 31 25 82 3 46 AnJo 83 10 43 3 46 lnqi 70 78 2 74 sbs 75 48 2 61 EExp 11 94 2 52 EBPI 68 0 2 34 ATCt 30 30 2 30 GK5 35 17 2 26 Rd11 6 6 2 6
538 is probably now 538R or a precedent form.
Moreover, on @Crow’s point about “unofficial” winners, kmedved’s DARKO projection did so last season (see: https://i.imgur.com/XDACgyj.png).
That, and DARKO also being Top 9-ish this year despite being relatively new may speak to the benefit of updating prediction priors with pre-season games data.
An “optimal” approach probably involves a combination of metric blending AND preseason adjustment.
Re: 2022-23 team win projection contest
Posted: Thu Mar 23, 2023 12:05 pm
It may be that prior MAE or RMSE better predicts how well an entrant will do in the upcoming season. It may be that an exponent between 1 and 2 is better; or less than 1.
Skimming over the table, it looks pretty random from year to year.
However, if anyone should just average the predictions of all apbr entries -- this "entry" is know as avgA this year -- they would almost certainly be in contention.
And without such a blatant shortcut, it's usually good to avoid "extreme" guesses. This year's leader was not highest or lowest on any team; they're just on the "right side" of most.
The closest dtka comes to an outlier would be Atl -- guessing 50, which turns out badly. Otherwise, mostly middle-of-the-pack predictions which are often good and never very bad, relative to everyone else.
Now tarrazu also had no highest or lowest guesses, and they're merely middling among "us"; still better than avg in the whole field.
I admire the bold prediction anyway. Crow and eminence continue that tradition.
But there's no absolute from year to year. Some years, teams just don't do what they're spozed to.
The worst prediction errors in 2017 would have been the best in other years:The "bottom" numbers are actually next-to-worst, as there are often outliers that either weren't serious or otherwise off the rails.
Skimming over the table, it looks pretty random from year to year.
However, if anyone should just average the predictions of all apbr entries -- this "entry" is know as avgA this year -- they would almost certainly be in contention.
And without such a blatant shortcut, it's usually good to avoid "extreme" guesses. This year's leader was not highest or lowest on any team; they're just on the "right side" of most.
The closest dtka comes to an outlier would be Atl -- guessing 50, which turns out badly. Otherwise, mostly middle-of-the-pack predictions which are often good and never very bad, relative to everyone else.
Now tarrazu also had no highest or lowest guesses, and they're merely middling among "us"; still better than avg in the whole field.
I admire the bold prediction anyway. Crow and eminence continue that tradition.
Yes, the amateurs and others here have done well against standards such as "vegas" most years.lots of good to great performances.
But there's no absolute from year to year. Some years, teams just don't do what they're spozed to.
The worst prediction errors in 2017 would have been the best in other years:
Code: Select all
year top mid bottom
2016 5.73 7.00 8.60
2017 3.84 4.20 4.93
2018 5.80 6.10 6.76
2019 5.87 6.60 7.88
2020 6.07 6.70 7.78
2021 5.47 6.00 8.33
2022 6.49 7.10 8.63
2023 5.06 6.00 6.98Re: 2022-23 team win projection contest
Posted: Fri Mar 24, 2023 12:13 pm
Code: Select all
. avg err rmse r^2 avg err rmse r^2
dtka 5.02 6.13 .64 BIPM 6.14 7.62 .63
avgA 5.42 6.40 .60 538R 6.26 7.79 .49
KPel 5.44 6.68 .54 22Re 6.60 7.97 .34
vzro 5.61 6.92 .53 TmRk 6.76 7.93 .48
DRKO 5.76 6.75 .55 vegas 6.76 8.00 .48
LEBR 5.83 6.52 .58 nuFi 6.92 8.25 .46
emin 5.90 7.37 .52 AnBa 6.93 8.21 .50
MPra 5.91 7.41 .61 EExp 6.94 8.32 .47
trzu 5.93 6.97 .58 4141 6.97 9.80
EBPI 5.95 7.35 .49 538E 7.58 9.09 .29
Crow 5.97 6.83 .59 2022 8.23 9.79 .34
ncs. 6.00 7.04 .52 Code: Select all
Atl w Bos w Brk w Cha w Chi w Cle w Det w Ind w
dt 50 vz 58 em 51 KP 41 nc 41 b-r 52 vz 31 KP 38
tr 48 tr 57 nc 47 vz 39 cr 40 em 50 nc 27 b-r 36
cr 47 nc 57 tr 47 dt 33 b-r 40 tr 48 em 27 nc 34
KP 46 b-r 56 KP 46 nc 32 KP 38 vz 48 KP 26 em 33
nc 45 em 56 b-r 44 em 32 dt 38 cr 47 dt 24 vz 32
vz 44 dt 55 dt 44 tr 32 tr 38 dt 46 tr 24 dt 31
em 44 KP 54 cr 44 cr 30 vz 37 KP 43 cr 23 tr 28
b-r 40 cr 53 vz 42 b-r 27 em 35 nc 43 b-r 18 cr 28
Mia w Mil w NYK w Orl w Phl w Tor w Was w
nc 51 b-r 58 b-r 46 b-r 34 b-r 54 cr 52 vz 40
tr 50 em 55 cr 44 KP 31 tr 54 tr 50 KP 40
cr 50 dt 52 vz 43 em 28 dt 53 nc 49 dt 38
vz 49 vz 52 nc 43 tr 28 em 52 dt 48 nc 37
em 48 tr 50 dt 43 cr 28 nc 52 KP 47 b-r 37
dt 47 KP 50 tr 42 nc 28 vz 51 vz 44 em 34
KP 46 cr 49 KP 42 dt 26 cr 51 em 40 tr 33
b-r 44 nc 48 em 36 vz 26 KP 48 b-r 40 cr 30
Dal w Den w GSW w Hou w LAC w LAL w Mem w Min w
nc 47 b-r 54 em 52 nc 27 cr 51 dt 43 cr 52 em 53
tr 47 cr 53 cr 51 cr 27 em 49 vz 43 b-r 51 tr 50
vz 46 em 51 tr 50 KP 27 tr 49 em 42 dt 50 dt 49
dt 46 dt 49 dt 47 em 24 vz 45 b-r 41 tr 48 cr 47
cr 46 tr 49 vz 47 tr 23 nc 45 nc 39 nc 47 nc 47
em 44 vz 48 nc 44 b-r 20 dt 45 tr 37 em 47 KP 46
KP 43 KP 48 b-r 43 dt 20 KP 44 KP 37 vz 47 vz 45
b-r 40 nc 47 KP 42 vz 18 b-r 43 cr 35 KP 46 b-r 41
NOP w OKC w Phx w Por w Sac w SAS w Uta w
cr 49 b-r 41 nc 54 em 40 b-r 49 vz 33 b-r 39
tr 48 cr 27 tr 52 tr 37 cr 39 KP 31 vz 39
KP 48 KP 27 vz 51 KP 37 tr 37 em 31 KP 35
nc 46 dt 26 em 50 b-r 36 dt 37 nc 31 nc 35
dt 46 vz 26 dt 50 dt 36 nc 37 dt 26 dt 32
vz 44 nc 25 cr 50 vz 33 KP 37 cr 25 cr 29
em 43 tr 24 KP 49 cr 33 vz 32 tr 25 tr 29
b-r 40 em 23 b-r 43 nc 28 em 32 b-r 21 em 28
Code: Select all
. avg err rmse r^2 avg err rmse r^2
dtka 5.07 6.15 .64 EBPI 6.06 7.43 .49
avgA 5.45 6.44 .60 BIPM 6.07 7.59 .63
vzro 5.59 6.99 .52 538R 6.27 7.86 .48
KPel 5.60 6.68 .54 22Re 6.62 8.03 .34
LEBR 5.72 6.46 .59 TmRk 6.78 7.92 .48
DRKO 5.81 6.79 .55 vegas 6.78 7.99 .49
Crow 5.90 6.80 .59 EExp 6.89 8.27 .47
ncs. 5.93 7.07 .52 nuFi 6.91 8.30 .46
MPra 6.00 7.44 .60 AnBa 6.92 8.17 .51
emin 6.02 7.47 .51 4141 7.04 9.88
trzu 6.04 6.99 .58 538E 7.67 9.17 .29
Re: 2022-23 team win projection contest
Posted: Sun Mar 26, 2023 5:08 pm
I have a lot of close or closest to BRef projection.
Some big misses too. Raptors and Clippers are among the bigger disappointments.
I am guessing for some individual improvement at the end (don't know about relative) but we'll see.
Some big misses too. Raptors and Clippers are among the bigger disappointments.
I am guessing for some individual improvement at the end (don't know about relative) but we'll see.
Re: 2022-23 team win projection contest
Posted: Sun Mar 26, 2023 9:45 pm
To @Mike G’s point about averaging projections similar to the avgA “entry,” it’s actually a consistent finding in the forecasting literature that the simple arithmetic mean (i.e., with equal weighting) of different forecasts often provides highly accurate overall predictions. Link: https://www.researchgate.net/profile/An ... iewer=true
Complex approaches to estimating the ‘‘best’’ combining procedures do not seem to matter that much. Link: https://faculty.fuqua.duke.edu/~clemen/ ... JOF-89.pdf
The unweighted average provides an efficient trade-off between precision and uncertainty.
Obviously, there is a systemic issue here that if everyone or most are using an average to make predictions, there would no or few models to include in that average — affects the forecast accuracy and diminishes the sought benefit.
Still, it’s all a fascinating idea to keep in mind.
Re: 2022-23 team win projection contest
Posted: Mon Mar 27, 2023 1:41 pm
My predictions are being seriously flattered by the MAE here, but to add the to prediction averaging chat: whilst I did not use any minutes projections at all for this, merely flattened average statistics at several roster sizes, I did average the predictions between those roster sizes. Some teams showed up as having a great 5 man roster, but lacked depth and as such were dragged down. Some were vice versa.
Re: 2022-23 team win projection contest
Posted: Tue Mar 28, 2023 2:53 pm
Could / would you say a little more about how you calculate "flattened average statistics" and the range of "roster sizes" included? Using mean or median?
You believe this gets at "talent" & depth. Ok. But actual coaching management of minutes is de-emphasized / eliminated. Does that suggest you more than others think coaches matter little relative to talent, at least as they currently coach and handle lineups & minutes?
You believe this gets at "talent" & depth. Ok. But actual coaching management of minutes is de-emphasized / eliminated. Does that suggest you more than others think coaches matter little relative to talent, at least as they currently coach and handle lineups & minutes?
Re: 2022-23 team win projection contest
Posted: Tue Mar 28, 2023 2:54 pm
I'm now 7th on average error and a little bit better still on the other two. Only 2 above me on average error are also above me on r2.
Re: 2022-23 team win projection contest
Posted: Tue Mar 28, 2023 6:54 pm
Of course, not all forecasts are improved by sheer number of forecasters. The ESPN experts I think numbered 15-20, but they aren't among the leaders.DarkStar48 wrote: ↑Sun Mar 26, 2023 9:45 pm ... it’s actually a consistent finding in the forecasting literature that the simple arithmetic mean (i.e., with equal weighting) of different forecasts often provides highly accurate overall predictions...
avgA is the error produced by averaging the predictions of our 6: emin, ncs, dtka, trzu, vzro, and Crow.
Just now I added each competing entry to those 6 and divided by 7, to see who else would improve (or worsen) the avg.
Further, I removed each of the 6 and divided by 5. From most to least improvement:
Code: Select all
err x impr err x impr
5.16 +LEBR .125 5.30 +22Re -.017
5.20 -trzu .090 5.32 +AnBa -.031
5.20 +4141 .088 5.32 +DRKO -.035
5.20 +BIPM .087 5.36 +538R -.074
5.22 +MPra .064 5.39 +nuFi -.101
5.22 -Crow .063 5.40 +vegas -.112
5.24 +KPel .046 5.40 +TmRk -.112
5.26 -emin .030 5.40 +EExp -.117
5.26 -ncs. .026 5.43 -vzro -.138
5.29 +EBPI .003 5.43 -dtka -.144
. 5.43 +2022 -.145
5.29 avgA .000 5.52 +538E -.236
Adding 538E is even worse than removing dtka.
There's some inverse correlation with individual entries' error sizes. But I suspect there's also a benefit from someone's predictions cancelling the recurring errors within our group of 6.
For example: BIPM isn't particularly great in the contest but offers these corrections: Our 6 were too high on Hou, Mia, and Det; BIPM had the lowest guess on them. We were well low on Sac and Mil; they were highest on those 2.
Re: 2022-23 team win projection contest
Posted: Tue Mar 28, 2023 7:48 pm
This is a reminder of how few were direct / official contest entries, fwiw. I am 3rd of 6 for that.
Re: 2022-23 team win projection contest
Posted: Tue Mar 28, 2023 7:57 pm
I think that coaching matters. I think allocating minutes to your best players, finding combinations that work well, keeping the locker room together and maintaining course on your plans matters. However, I think the impact a coach has tends to show up on the court relatively quickly, so player data begins to capture it. I think in the playoffs the impact of coaching is greater. But do I think talent is most of the equation? Yes. As long as your roster looks like it can provide enough of the essential elements of basketball (i.e. a reasonable distribution of offensive and defensive abilities) then I think talent is what will decide things more than anything else, so allocating minutes to that talent is priority number one.Crow wrote: ↑Tue Mar 28, 2023 2:53 pm Could / would you say a little more about how you calculate "flattened average statistics" and the range of "roster sizes" included? Using mean or median?
You believe this gets at "talent" & depth. Ok. But actual coaching management of minutes is de-emphasized / eliminated. Does that suggest you more than others think coaches matter little relative to talent, at least as they currently coach and handle lineups & minutes?
Regarding details of how I calculated the features for these models, it went something like this:
Code: Select all
1.) Gather roster.
2.) For N from 5 to 9 do:
a.) take N players from roster with most allocated minutes in previous year.
b.) average the player statistics (advanced box stats, other stuff I have developed) across those N, without weighting by anything (hence a flat/unweighted average). Each averaged statistic becomes a feature in the model.