APBRmetrics

Hi All,


New post is up where I look at using a simple corporate style report format to help solve the issue of communicating data from multiple sources (sportsVU, analytics staff, scouting reports etc) all on one easy to read page.


check it out at www.zigzaganalytics.com

If the problem in the NBA is that the "analytics people" and the "non analytics people" are not communicating, does it really make sense to test this presentation (which is ostensibly targeting "non analytics people") with a bunch of NBA analytics people?

I'm speculating a bit, but I suspect that the issues are more fundamental than presentation. Specifically, that the history of numeracy in sports is more about results-oriented trivial than about evidence-based decision making, and that the problem issue has as much to do with making things trustworthy and practically actionable as it does with making them easily comprehensible.

I guess the purpose of creating a "1 pager" which you can read about the origins by googling "Toyota a3" or "Toyota 1 pager" is that the report is just an output of gaining consensus before hand between in this case coaches and front office staff which incorporates analytics teams. If an NBA team is going to be successful with using analytics there can't be this stigma of "analytics people" and "non-analytics people", its 1 team who have agreed as a strategy to use data to help make decisions.

All the pre work and meetings to discuss what will work/what wont, what's important and what's not is done before hand where everyone works together to gain consensus. The report is an output of many meetings and sources of data and is supposed to bring it all together to 1 simple spot where its very clear to all what the message and direction of the team is.

It's not to say no analytic work has been done just because there aren't loads of graphs and analysis, all that work has been done and filtered through already.

So in creating a 1 pager the team (coaches, front office) will inherently communicate and collaborate more which is where the "trust" piece comes into play and is built from those meetings of gaining consensus.

watto84 wrote:I guess the purpose of creating a "1 pager" which you can read about the origins by googling "Toyota a3" or "Toyota 1 pager" is that the report is just an output of gaining consensus before hand between in this case coaches and front office staff which incorporates analytics teams. ...

To me, that "one pager" in the article looks quite different than the sort of thing that comes up when I google "Toyota A3". A3s seem to be much more focused, and incorporate "Effect Confirmation" and "Follow-up" sections which are IMO really important.

... If an NBA team is going to be successful with using analytics there can't be this stigma of "analytics people" and "non-analytics people", its 1 team who have agreed as a strategy to use data to help make decisions. ...

The thing about stigma is true, but as long as team members are capable and trust each other, it shouldn't be a problem if various people have different decision making strategies.

The format of the Toyota example is very different yes but its the underlying concepts I was trying to mimic with creating a 1 pager. That being, gaining consensus with everyone involved, bringing together all important findings to 1 place and the report really being an already aligned message that everyone already agrees on as they have been involved with the process a long the way and understand the goals.

The format I used is actually an exact replica of what I use for IT project updates at MARS inc where I work today.

I agree with you. One pager filtered stuff would be a nice step towards creating a mutual trust but I also have to agree with Nate that everything ultimately depends on the result. The thing is nobody's gonna care about how analytics people come with those numbers or how nice and pure and short they present the data. Instead there will be simple questions like "How's the minute distribution should be? Which players should we target? Whom should we trade? Whom should we draft? How many years will it take us to get the championship?" And weekly there can be questions like "Coach says we have beeen suffering at getting defensive rebounds lately and that's the reason we've been losing lately, does your "stuff" agree?

I never worked for any sports team staff so I don't know if my imagination is closer to the truth or not. However, there are plenty of people here that worked for or is working for an NBA team here so they may share insights if they choose to.

What you are refering to looks pretty much like the concept of a scientific poster. Toyota seems to have adapted that concept, nothing more.

In itself the problem of communicating results to an audience which does not have the same scientific background exists. A similar thing will very likely occur for NBA teams, because of the different educational backgrounds. Someone who is familar with numbers will have a different understanding than someone who does not have such a background. Presenting research or reports in an easy-to-read way must be based on a common ground. Finding that common ground isn't an easy task at all, and as you pointed out, will likely make creating such reports difficult.

Anyway, what I see on your blog looks not like a good solution at all, because you basically decreased the font size in order to fit too much information in a difficult-to-understand way on one page. Nothing else. What people grasp easily are figures which can transport the information based on a common ground; a common ground which is specific for the audience. In case of a NBA team using basketball terms or presenting the information in figures basketball people are used to, is the way to go. Maybe try looking up the work by Kirk Goldsberry to get an idea what I mean.

Also, scouting reports will likely differ depending on the person having to read those reports. Handouts for players will likely have to look different than for coaches, and those will likely differ from the ones generated for front office people, because each person will have a different background as well as a different way of using such reports.

“If you can’t explain something on 1 page, you haven’t thought about it enough” I find such statement to be rather stupid, because the level of detail as well as the complexity of the subject will dictate the length much more than "how long I thought about it". As an example I can explain the basic ideas of the GTR with a few sentences or using a few figures (in fact I can use only one figure), but nobody will be able to run calculations based on that. It is not that I haven't thought about it enough or that I don't understand the GTR, but that the underlying math is just simply nothing which can be put on one page (unless I reduce the font size so much that someone would need a microscope to read it).
The other issue is: Is it really necessary to put it on one page or would it be better to have the information spread out to 3 pages instead? Such thing depends heavily on the preferences of the audience. Another thing to consider here is the way how such information is presented. Are those printouts or does the coach want that on his tablet? Do the players prefer scouting reports for the next opponents to be readable on their phones instead in a twitter-like style?

watto84 wrote:.... That being, gaining consensus with everyone involved, bringing together all important findings to 1 place and the report really being an already aligned message that everyone already agrees on as they have been involved with the process a long the way and understand the goals....

I don't want to put words into anyone else's mouth, but this makes it seem like you think people will magically agree with something just because it's printed on a single piece of paper. In my limited experience, consensus and trust are things that are established in other ways.

mystic wrote:... I can explain the basic ideas of the GTR with a few sentences ...

Are you referring to the Nissan GTR or something else?

Nate wrote: Are you referring to the Nissan GTR or something else?

GTR = general theory of relativity or shorter known as GR (general relativity)

There is a place for a 1 page summary. This form of status report is ok but from a quick look at a MIT article about A3 style it doesn't seem fully up to the A3 standards of defining the problem and plan, issues and strategies and presenting it in a tight logic flow. Yes, there are goals to be pretty good to make playoffs, pretty good at offense, defense and pretty good at every stat and I know this is just an example but I'd think a real plan might be a bit more nuanced, recognizing strengths & weaknesses they are working with and maybe showing more than generic strategy to be pretty good at everything.

While accepting the place of a top page- perhaps at a higher strategic pitch- I think that one of the biggest issues in sports analytics maybe coaches and players being unwilling to read, think about and offer thoughtful feedback on 5, 10 or 20 pages of details, details about strategy, performance, correction, methods of correction to their fullest capacity daily (not just a skim and a set down with or without an eye roll) and to work as hard at getting better at understanding and using the analytics as they do any and every other part of preparing. This is a complicated game and there is plenty meaningful to say, read and discuss. There are handfuls or dozens of risks, issues and opportunities much of the time. Perhaps you only highlight one or two of the biggest at a time but you need a report with the full list to be seen and discussed before selecting the highlights. A summary top page is a point of focus but the is no short cut to avoid thinking about a lot of details about the plan, problem, etc. along the way to acting appropriately. You already agreed there is a lot of detail that comes before this one pager, so we don't disagree about that process conceptually but I wanted to emphasize the importance of doing all the work to get to the 1 page because I fear too many want to skip to it instead of build to it constantly.

the history of numeracy in sports is more about results-oriented trivial than about evidence-based decision making

i disagree...

the problem issue has as much to do with making things trustworthy and practically actionable as it does with making them easily comprehensible.

i agree...

If an NBA team is going to be successful with using analytics there can't be this stigma of "analytics people" and "non-analytics people"

oh yes there can...

its 1 team who have agreed as a strategy to use data to help make decisions.

in an ideal world...

All the pre work and meetings to discuss what will work/what wont, what's important and what's not is done before hand where everyone works together to gain consensus.

opinions vary all the time, there is rarely a single consensus. decisions are made on players on a daily basis (who to draft, who to trade for, who to sign as a free agent, who to get for the d-league affiliate, who simply to work out in house) with input from many sources and not everyone is involved in every decision or are around to be involved in every decision. heck pre-game planning and opinions on how to defend certain teams and players (or by whom) vary all the time...

The report is an output of many meetings and sources of data and is supposed to bring it all together to 1 simple spot where its very clear to all what the message and direction of the team is.

again in an ideal world, but rarely in a corporate setting do you get this, let alone with an nba team. that is why you have decision makers - to try to make sense of the varying opinions he hears and data he receives from the experts he has hired...

It's not to say no analytic work has been done just because there aren't loads of graphs and analysis, all that work has been done and filtered through already.

on the contrary it's a never ending continual process - there's always new information, changing information, old information that was overlooked, etc...

That being, gaining consensus with everyone involved, bringing together all important findings to 1 place and the report really being an already aligned message that everyone already agrees on as they have been involved with the process a long the way and understand the goals.

if you are talking about a process such as bringing a product to market, a process involving engineers, designers, planners, accountants, marketers, etc. i can see this logic. but in the nba decisions that can have profound impact on a team's future are made on a seemingly daily basis...

the long term goals of the team can be agreed upon and followed. but one must always adapt to a constantly changing situation. you can over time do all the legwork and due diligence to target, acquire, sign, and play a top player, and yet have to scramble when he is lost for the season to injury while chewing up 1/5 of your salary cap...

what do you do then? you have limited resources and limited time to adapt to salvage the season - if salvaging the season is the goal, and it may not be if salvaging the season may disrupt carefully considered longer term plans...

just look at new orleans and memphis this season, to name a few...

do you pick up the recently released veteran to bolster your roster even if it puts you over the cap? if you don't and he goes elsewhere and the very next day your star gets injured and is out for the season, were you at fault for not grabbing the guy when you could?...

If the problem in the NBA is that the "analytics people" and the "non analytics people" are not communicating, does it really make sense to test this presentation (which is ostensibly targeting "non analytics people") with a bunch of NBA analytics people?

or how about how do the non-analytics people feel when their own analytics people can't agree on something? do they then fault the analytics people because they can't agree among themselves, or fault the analytics process itself and thus cast doubt on their methodologies?...

demarcus cousins makes $16 mil this year, $17 and $18 the next two years - that's a huge investment...

he's just 25 years of age and is scoring 27 pts/g and grabbing 11 reb/g. the only other players to do that in a season are players like moses malone, shaquille o'neal, and hakeem olajuwon - all-time great Cs. some think cousins is deserving of all-NBA 1st team...

yet his team is not good. he shoots poorly on 2s for a C - that's more missed shots rebounded by the opponent - plus he commits alot of turnovers...

if i told you he was not very good, and not worth his salary, despite many thinking he's all-NBA quality, and the fact that those with similar per game scoring/rebounding numbers are all-time great Cs, what would you say?...

or if i told you kawhi leonard now is as good as lebron james ever was?...

GTR = general theory of relativity or shorter known as GR (general relativity)

it' be nice to marry general relativity and quantum mechanics, and show it on one page (or one t-shirt)...

bchaikin wrote: it' be nice to marry general relativity and quantum mechanics, and show it on one page (or one t-shirt)...

That would be extremely nice ... maybe a picture of a Calabi-Yau manifold for the shirt ... oh well, that would be a pretty bold statement, I guess