(Adjusted) Impact on Win Probability

[permaximum]

I know that guy's posts are very rough and don't explain much but that's what I got from it.

Actually, J.E. explained it very well in the first post. But I guess you got a little bit confused here, because I answered a question about the interpretation, while giving an example of interpreting the results differently at the end (as percentage instead of percentage points), which wouldn't make much sense.

Here is my first post which xkonk replied to: viewtopic.php?f=2&t=9114#p26838

The interpretation would actually be that a player would raise/lower the win probablity over average by x percentage points.

https://en.wikipedia.org/wiki/Percentage_point (just in case, you don't know what percentage points means)

I know what it means. I exactly meant win probability chance. I'm saying in your example it would translate to 63.2% of winning the game. Not 56.6%. It's additive. Green will always add 0.132 pps when he replaces an average player regardless of the team's total winning probability... So in your example Green would increase that teams winning chance by 26.4%. If he would replace an average player from a team with 10% wp (it's not possible to have it this low according to the table in the OP), he would increase their chances of winning by 130% thus making it 23% in the end.

Either you got me wrong or you're telling it doesn't work like that and Green would increase "percentage" of winning by 6.6% for a team with 50% of wp.

[xkonk]

Now, at what point could the math lead to weird results? 108% doesn't exists at all, but comes from a misinterpretation of the results. Taking the results from J.E. for Green for example, simply means that with Green playing all 200 possessions (100 offensive and 100 defensive possessions) of a theoretical game with such an amount of possessions, that on neutral court Green plus average players would win 63.2 % of the times against a team full of average players. That's it.

Since I provided the misinterpretation, let me be more explicit and someone can tell me where I went wrong.

A team's probability to win a game is dependent on a) time left in the game, b) current lead (can be negative), c) whether they have the ball and d) the players on the court.

To give an example: If your team is down 2 with 1s to go (a scenario where you're unlikely to win), and you hit a 3 as time expires you changed your team's probability of winning the game from <50% to 100%
On the other hand, if you're up 2 with 1s to go, and hit a 3, you only moved your team's probability of winning from ~>95% to 100%
Somewhat obviously, it is more important for your team to go on a run in a tight game, vs when being up by 20+

One can use the standard APM framework to carry out the analysis, but the y-vector - filled with points-per-possession in APM - gets replaced with changes of win probability.
Thanks to the APM framework we can then estimate each player's influence on win probability, controlling for factors a)-d) from above - most importantly controlling for who you're on the court with.

J.E.'s running a ridge regression as opposed to a standard one, but we can set that aside. He's trying to get the B's that best fit an equation that looks like (change in win probability per possession) = B0 + B1*(time in game) + B2*(current lead) + B3*(possession of ball) + (lots of Bs times player on/off court). Once the regression provides the Bs, if you want to get the regression's estimate for some situation you just enter all the appropriate factors - players on court, time in game, etc.

When J.E.'s table says that Draymond is 13.2 per 200 possessions, I assume that at the least, that means that the B for Draymond is actually something like 0.066 per possession, and J.E. has multiplied the output by 200 to give a more readable number. That isn't especially important. My understanding of how to interpret a regression coefficient is that you say that changing the value of an X by 1 is expected on average to change the value of y by the appropriate B holding all other Xs constant. In J.E.'s example, if the time in game is 1 second, the lead is 2, and possession is "your team", and a group of average players going against another group of average players, the y is something like 5 if your team hits a 3 (the ~100% - ~95%). But say that before hitting the 3, they call time-out and substitute Draymond in. Now they hit the 3. Instead of adding 5%, they should add 5+.066 = 5.066%. So that was a mistake on my part; the example in my post before shouldn't have added 13% because it wasn't over 200 possessions.

But, I'm still not sure that you interpret it as '13.2% over an entire game'. It's over 200 possessions at some particular time left and lead, right? If the team in my example above played 200 games where they substituted Draymond in before hitting a 3 with 1 second left and a 2 point lead, they would gain 5.066% instead of 5% 200 times. And you could still have some odd results, like if the team were to substitute Draymond in for defense after the 3 instead of for offense before the 3. They would go from 100% to 100.066%. If you had one of the Warriors' best lineups against an average group in the same circumstance, you could get up to about 100.2%. Right?

[mystic]

When J.E.'s table says that Draymond is 13.2 per 200 possessions, I assume that at the least, that means that the B for Draymond is actually something like 0.066 per possession, and J.E. has multiplied the output by 200 to give a more readable number.

You can get that output directly, if you multiply the value in the response vector accordingly. We (I assume most do the same here) are using Net ratings for RAPM too. But as you said, that isn't really important.

But, I'm still not sure that you interpret it as '13.2% over an entire game'. It's over 200 possessions at some particular time left and lead, right? If the team in my example above played 200 games where they substituted Draymond in before hitting a 3 with 1 second left and a 2 point lead, they would gain 5.066% instead of 5% 200 times. And you could still have some odd results, like if the team were to substitute Draymond in for defense after the 3 instead of for offense before the 3. They would go from 100% to 100.066%. If you had one of the Warriors' best lineups against an average group in the same circumstance, you could get up to about 100.2%. Right?

I understand what you want to say, but without the equation needed to calculate the necessary win probability at this point in the game, the discussion is really just a theoretical scenario, which just has little to do with the reality, from my perspective. You are correct, that theoretically speaking with a split second left (< 0.3 sec) and a lead bigger than 3, a team would have 1 anyway (at least according to my used formula which does not take possession of the ball into account; I haven't thought about that at all, but the issue might just be an inadequate equation for calculating the win probability, and I have to emphasize again that at least in my case that is just an approximation). Now, you have to take into account that there are 10 players on the court and the difference between the sum of each team would determine whether we raise or lower that. Theoretically, we could indeed end up with a value slightly above or below 1 for that scenario.
I could simply use a trick here, where I set f(x) > 1 to f(x)=1 and be done with it. That isn't an elegant solution, but it is easy and works.

Edit: Somehow the first part of my response didn't survive the submit and I had to add it now.

[Nate]

...
I could simply use a trick here, where I set f(x) > 1 to f(x)=1 and be done with it. That isn't an elegant solution, but it is easy and works.

The traditional method is to do something like:

sigma ( sum ( logit (x) ) )

(Where sigma is the logistic function.)

Though I'm not sophisticated enough to have a sense about the particulars, I believer that there are profound and subtle issues that can come up when you apply linear regression to something non-linear like WPA.

[J.E.]

For example, for determining win probability for a situation with, say, 39:15 left and up 10, I can't just look up all occurances with the exact same time and score difference to get an expected win% - the sample would simply be too small and the noise would be huge.

I used 15 sec intervall, which would roughly mean that at 96 Pace each possession would be covered separately. That means I had 192 "buckets" with an actual scoring margin aligned to W/L (1/0) for the home team; OT games were ignored. As raw data I used the bbv dataset from 2006 to 2011 (playoffs included).

An obvious issue is the missing information which team has possession of the ball. And I did not test whether using the OT games as well would have an effect.

The intervals I use grow with size the further away from "0 seconds left".
Instead of using fixed intervals I think it might make sense to do a "nearest neighbour" analysis. For example, one could increase current search window by 1 second (to the left and right) until the sample size has reached X (maybe 100?)

OT games I currently score as half a win

What I need is the win probability before and after the possession. Then I take the difference of those two probabilities and write that difference into the response vector instead of the result of the possession. The design matrix doesn't get changed.

This is correct. The results vector contains the changes in probability

As for the interpretation of (Draymond's) coefficient(s). I think the resulting coefficients give an estimate of what will happen in an average situation - or, what will happen in the average situation this player was put in.

To a degree, I think, it also means a very good player's WPA could potentially suffer from being on the court lots when the game is already won. He can't add to the 100% Win Probability the game is already in. The result for those observations will be 0 each, which is less than his usual average

[Nate]

...

To a degree, I think, it also means a very good player's WPA could potentially suffer from being on the court lots when the game is already won. He can't add to the 100% Win Probability the game is already in. The result for those observations will be 0 each, which is less than his usual average

I commented on that earlier - depending on how significant this kind of contextual effect is, the WPA assessment might be more of a proxy measurement of the coaches' assessment of the player than anything intrinsic about the player.

Suppose that a coach only puts in the starting line-up when the game is close, or when the team is behind, and puts in bench players when there's a big lead. Even if the scoring were just a balanced random walk, the starters would end up with higher WPA than the bench players.

[mystic]

The intervals I use grow with size the further away from "0 seconds left".
Instead of using fixed intervals I think it might make sense to do a "nearest neighbour" analysis. For example, one could increase current search window by 1 second (to the left and right) until the sample size has reached X (maybe 100?)

That makes a lot of sense. I thought about using "possessions left", which would have solved my OT games dilemma. If specific "possession-left" values don't reach the needed sample size, they could simply be eliminated. No idea what X should be.

As for the interpretation of (Draymond's) coefficient(s). I think the resulting coefficients give an estimate of what will happen in an average situation - or, what will happen in the average situation this player was put in.

Well put.

To a degree, I think, it also means a very good player's WPA could potentially suffer from being on the court lots when the game is already won. He can't add to the 100% Win Probability the game is already in. The result for those observations will be 0 each, which is less than his usual average

I'm not really seeing the issue here, because in reality the player doesn't help his team win more games by being on the court during "garbage time". It might have been a coaching decision, but usually coaches are subbing their best players out anyway.


Nate, I understand your point and it is a good one in general, but from my perspective it isn't really that relevant. What it basically comes down to: For the most situation the interpretation fits good enough, but for some specific situations the interpretation obviously doesn't work.
And the linear regression is used to solve such a problem: http://www.apbr.org/metrics/viewtopic.php?f=2&t=8239
The design matrix is the same as for RAPM, just the values of the response vector are getting changed to win probability changes instead of the Net rating (or when accounting for Off/Def the respective Ortg/Drtg). I don't really see the issue here. In a sense we can imagine that we would get similar RAPM values (if we break that down to z-scores at least), if we would use a sophisticated nonlinear weighting scheme, which would account for the actual scoring margin and the time left in the game. That's actually the very reason I used that way, because the weighting schemes I tried before which were supposed to account for example for a possible "clutch" factor, just didn't give me an improved out-of-sample prediction. So, when I stumbled over Neil Payne's blog post I had the idea, that using win probabilities instead would solve the dilemma.

[mystic]

Suppose that a coach only puts in the starting line-up when the game is close, or when the team is behind, and puts in bench players when there's a big lead. Even if the scoring were just a balanced random walk, the starters would end up with higher WPA than the bench players.

That is true, but we can also assume that coaches know what they are doing, thus using the better players at the end of a close game is actually based on intrinsic values of the players the coaches deem to be the better players. Also, the effect would go both ways, because the used players would have a bigger negative effect, if the team's win probability is decreasing. So, players who can constantly outperform expectations during that time, will end up with the higher rating, the others with a lower value. The question should be: Does that method perform better in a out-of-sample test (using the same conditions like prior information for both)? From my results, it does better at least for the prior informed version. It predicted the correct winner more often and has a slightly but significant better RMSE (converting win probabilities into Net ratings). That was not done in a retrodiction test, but real predictions using a minute distribution and pace prediction model ahead of the games. The sample contains basically all games played between December and the finals in each of the past nearly 3 seasons (obviously, the current season isn't over yet) with the exception of a few games around Christmas time to the first couple of days of January for 2013/14 and 2014/15. Well, I'm really interested to see what J.E. gets in his out-of-sample testing (assuming he will do that later on).

[tacoman206]

Couldn't we just use something like WPA/LI to adjust for the effect of playing a lot when the game is already decided? Seems like it's a pretty simple thing to do for baseball games.

[J.E.]

Couldn't we just use something like WPA/LI to adjust for the effect of playing a lot when the game is already decided? Seems like it's a pretty simple thing to do for baseball games.

This needs to be tested but I fear it'll introduce more noise to the system. These late game situations as very noisy, as is. And they have a rather large influence on the coefficients because the swings can be very big (and remember we're trying to minimize squared error). If I understand WPA/LI correctly it'd give those late game situations more weight. That might potentially just blow up that noise

[Nate]

Couldn't we just use something like WPA/LI to adjust for the effect of playing a lot when the game is already decided? Seems like it's a pretty simple thing to do for baseball games.

This needs to be tested but I fear it'll introduce more noise to the system. These late game situations as very noisy, as is. And they have a rather large influence on the coefficients because the swings can be very big (and remember we're trying to minimize squared error). If I understand WPA/LI correctly it'd give those late game situations more weight. That might potentially just blow up that noise

That seems like it should do the opposite. WPA/LI should be something analogous to a leverage weighted linear stat. In other words it should be "roughly proportional" to points scored.

If you look at individual stints. We can think of WPA as (net points * win expectation per point) and LI as (win expectation per point) so you end up with WPA/LI as a proxy for (net points).

[ampersand5]

With this development + an additional year of PTPM in the database, this summer might bring about a much improved all-in-one metric.

I'm excited to hear any updates J.E., Mystic and anyone else contributing to this.

[Nate]

With this development + an additional year of PTPM in the database, this summer might bring about a much improved all-in-one metric...

Is there a basic reference on the PTPM methodology?

[ampersand5]

With this development + an additional year of PTPM in the database, this summer might bring about a much improved all-in-one metric...

Is there a basic reference on the PTPM methodology?

counting-the-baskets.typepad.com/my-blog/2014/09/introducing-player-tracking-plus-minus.html