(As an introductory note, it's been a while since I posted here, but a lot has happened in the interim, not just on the ice during the 2014-15 season but especially off of it, with the Summer of Analytics, the emergence of new statistical resources, and some interesting perspectives on goaltending from Steve Valiquette, Chris Boyle, and others. On top of all that, I was also inspired by attending the Ottawa Analytics Conference and meeting many of the people I have worked with and interacted with online. All that means I'm planning to get back into hockey analytics, which for the time being will include contributing to the Hockey Prospectus Annual and blogging in this space. Although I was pretty fortunate in having my last post here be on a topic that makes me look pretty good in hindsight (Dubnyk vs. Scrivens), it's long past time to starting pushing that one down the page.)
One of the things I have been thinking about lately, and something I wanted to bring up in my first post back, is the issue of career stats and arbitrary endpoints, to borrow a term used by ESPN sabermetrician Keith Law. Here's the description from
Law's glossary:
#arbitraryendpoints: Also known as cherry-picking, this means choosing one or both endpoints on a series of games to try to analyze a player. I’ve argued that it’s not arbitrary if the endpoint is tied to something specific, like a change in mechanics, an injury, or a recall from the minors, but even so, it’s always dangerous to throw out any data when you want to draw a conclusion.
I definitely agree that it is dangerous to throw out data and that it should never be done for the purposes of supporting an already-reached conclusion. However, I also agree that it is not always arbitrary to look at splits and segments of data rather than relying on the complete sample if there is good reason to expect that some of the data is not representative of an athlete's true talent level.
There seems to be an increasing trend on Hockey Twitter for people to simply pull up a goalie's career save percentage or career EV SV% and use that as the final verdict on their talent level. This can certainly be appropriate some of the time, perhaps even most of the time. I still think that goaltending is above all else a results business, and that statistical measurements remain very powerful methods of evaluating performance. That's why save percentage over a large sample size is usually a good proxy for a goalie's talent level. However it is still only a proxy, and the three points mentioned by Law do also apply to goaltenders. In the same way that Law might discount a pitcher's performance in his first year back from an arm injury, we might have reason to believe that a goalie could be deviating from his historical average because he is not yet at 100% health after coming back from some time off or has made some changes in his game after working with a new goalie coach.
On top of those factors there are others more specific to hockey goalies, such as team effects/shot quality, situational performance (e.g. EV vs. PK), home scorer bias/road performance, usage, etc. These factors generally do not have a major impact on stats, but margins are so slim in goaltending that even a slight advantage or disadvantage can have an effect on the rankings. Overall, I think sometimes insight can be missed by looking only at the big picture, and in those cases it is appropriate to take a deeper look. And I'll start doing that by focusing a magnifying glass on the short but interesting pro career of Jacob Markstrom.
