Kevin's blog has moved! Please read his work at http://examiningbaseball.weebly.com.
2013 MIT Sloan Sports Analytics Conference
On March 1st and 2nd, I attended the MIT Sloan Sports Analytics Conference at the Boston Convention Center. The conference had a variety of panel topics such as baseball and basketball analytics, and unique panels like data visualization, eCommerce, and the business of sports. The panelists are all well spoken and have extensive knowledge of their field. Overall, there were three main points that resonated throughout the conference that could be applied into many aspects of analytics. These were: the method of expressing the data is almost as important as the data itself, separating the “signal” from the “noise” is critical, and - most importantly - focusing on the process more than the outcome.
The Data Visualization panel was united in the belief that one of the biggest issues in analytics is communication of data. Paraag Marathe, the COO of the San Francisco 49ers, expressed his focus on data communication by saying that moderate-level analysis with excellent communication is better than great
analysis with minimal communication. Common data visualization tools today range from as basic as a basic picture or diagram to as complex as an interactive graph. The method of presenting the data varies based on the target audience. For example, if a coach is trying to communicate to a point guard that he needs to close out quickly on a certain player, then showing him an image of what he has to do is better than rattling off stats to him. On the contrary, if an analyst for a sports team was presenting their findings to a COO that has deeply rooted beliefs in analytics, then a more complex visualization may be
appropriate. However, too much “frill” in a visualization makes it difficult to understand and the message may be blocked by the complexity. It is important to
highlight the main points and stick to key information in a visualization. If the data collector does the storytelling, then the information will reach a wider audience. Again, this may also be taken to an extreme: Too little information may leave people unsatisfied and skeptical.
Separating the “signal” from the “noise” might be one of the hardest things to do in sports analytics, especially in sports that have fluid possession changes
like hockey or basketball. Nate Silver is the mastermind who has developed and applied this idea. So many factors can go into one play, and if the focus is on
one player, then the other player’s performance will impact the the focus players performance. In football, this happens all the time. If a running back
breaks a 70 yard touchdown run, so much more goes into the play than the running back's skill. Offensive line blocking, receivers blocking on the second level, defensive play call, and blown assignments all factor into the one play. There are two main ways to distinguish between the “signal” and the “noise”, these being using the data available and a person making a gut call. Using these two together can help make good decisions. A balance is also key between the two, as each has their strengths and weaknesses. Data is good a quantifying things and expressing them in a definitive way that a gut call cannot. With gut calls, people can ask questions and dig deeper, and support their decision on a personable level. In baseball, where immense amounts of data are collected because of the discrete nature of the game, analysts can dig deeper into data. Advanced statistics help minimize noise, like when xFIP and FIP can be used to get a clearer picture of a pitcher’s ability, compared to wins or ERA, which has more “noise” than FIP. Overall, a decision should never be based on just statistics, but instead a balance of stats and human gut calls, which help minimize, not eliminate the “noise”.
To a fan or a media outlet, the process is not something of value. The outcome is very important, if not the most important feature of an organization. If the
process is stellar, but the team ran into bad luck, and did not make the playoffs, then the media will rip the team apart without acknowledging the process. Each team has their own unique process, whether it be for scouting players, making trades, in-game decisions. The two key aspects of process are consistency and comparison. Consistency is key in process because it makes evaluating the outcomes of these processes possible. With no consistency, there are too many variables to control, and evaluation becomes a guessing game of what caused what to happen. Consistency also allows a process to be fine tuned to address any issues with it; reactionary decisions in an inconsistent process may create another issue or not solve the original issue. Comparison also ties into consistency. With a consistent process, it can be compared to other processes and evaluated for strengths and weaknesses. With the changing nature of the game and new data being collected every day, processes need to be adjusted to fit the evolution of the game. To further evaluate process, the
correlation of certain decisions and the results that they produce can be examined to help develop and improve the system. Certain statistics may have correlation, like team on base percentage and runs scored in baseball. Isolating certain players to scout with this statistic may lead to finding undervalued players that fit their system. Focusing on process may be difficult for fans and the media to do, but in the big picture it can lead to a long term success for a team.
Stats used in this blog are from a variety sources, especially ESPN and Pro Football Reference.