His presentation had the weighty title of “How Analytics has Limited the Impact of Cognitive Bias on Personnel Decisions,” and it lasted all of about 25 minutes in a small room at the Boston Convention Center on Saturday morning.
What it really was was the Toronto Maple Leafs finally allowing a peek behind the curtain, with assistant GM Kyle Dubas serving as the guide to how the richest team in hockey has changed.
Yes, they are using analytics, he told the audience at the Sloan Sports Analytics Conference.
No, they aren’t surprised they haven’t had instant results.
“There’s teams that are so far even ahead of us in our own league,” Dubas explained after his presentation, which outlined five biases – confirmation bias, recency bias, information bias, sample size bias, simplicity bias – that using statistics have helped him avoid. “And we’re trying to play catch-up as fast as we can. You’ve got to watch it because if you try to rush it you stumble and fall.”
The Leafs may not have been rushing, but their thought process has certainly undergone a rapid overhaul the last seven months. New president Brendan Shanahan made hiring Dubas his first dramatic move in late July last year – after the draft and free agency – and the culture within the franchise has slowly evolved from there.
GM Dave Nonis remains onboard but as a willing mentor and pupil, someone willing to teach but also to learn what Dubas – and his growing team of analysts and counters – see in hockey’s new numbers.
After years of being represented at Sloan only by former GM Brian Burke, who offered his annual griping about analytics routine on the weekend, the Leafs had five staffers in attendance – the most of any NHL team and more than most professional teams period.
The difference in the messaging between Leafs management, old and new, couldn’t be more stark.
“The notion that you can sit behind a computer and find athletes is bullshit,” Burke grumbled during his time on panel with Nate Silver, among others. Later, he added “this is still an eyeballs business.”
“What analytics taught me is your eyes and your mind are lying sons of bitches in the worst absolute way,” Dubas said earlier in the day, directly refuting Burke’s eyeballs adage.
Sloan is always about a lot of numbers. But where there’s universal agreement between everyone is that winning is the end goal, and wins have been hard to come by in Year 1 under the new regime in Toronto.
That has bred skeptics – the kind that fill Dubas’s Twitter feed after every loss.
Much of his talk at Sloan centered on his three years with the Soo Greyhounds in the Ontario Hockey League. Dubas explained some of his early failures and eventual success of using analytics there to turn a basement dwelling team into a contender in relatively short order.
The parallels with the Leafs were obvious – and it was hard to not read into his comments when he discussed the transformative effect a coaching change had with the Greyhounds.
The Leafs made a similar move in firing Randy Carlyle early January but have remained in freefall (4-16-2) since despite an uptick in analytics like possession (from 44.2 to 48.5 per cent).
The implication was that that will change and the results will come.
“What I learned was the extreme value of the buy-in throughout the organization,” Dubas said as he presented a graph of the Greyhounds’ dramatic improvement in possession data after hiring Sheldon Keefe as coach. “From the president down to the manager to the coach and scouts… You’re going to have a lot more success than if you have one person on your staff alone saying this is important.
“If you have a coach that isn’t really buying into it and you’re bringing in personnel that are analytics-friendly and you have a coach that isn’t, it’s not going to work, because they have to work together. They can’t work separately.”
The Leafs now have that buy-in, especially where it’s most important. Shanahan believes hockey’s data movement will help them, and he’s greenlighted the use of considerable resources for new hires and technology this season.
Shanahan has also pushed Dubas to explain his conclusions better, including a recent presentation to Leafs ownership where he had to outline the role variance – by way of a stat called PDO – has played in their recent woes.
“You’re paying your staff millions of dollars,” Dubas said of his owners, led by Bell and Rogers. “You don’t want to hear [the results] are influenced by luck.”
Beyond the buy-in, the second part of Dubas’s message at Sloan was that this transformation will also take time – especially for the Leafs. In Sault Ste. Marie, any mistake was washed away in two or three years because, by its very nature, junior hockey is cyclical. Players enter at 16 or 17 and quickly age out.
The time frames involved are much longer in the NHL, and mistakes can be more punishing – even with get-out-of-jail-free cards like the one Toronto lucked into in dealing David Clarkson’s contract on Thursday.
What that means is it could take years – three, four, or even more – to see the real results from the Leafs pursuit of this analytical path.
They’ve made major strides, but they’ve also only just begun.
“I think it’s going to take longer here,” Dubas said. “In the OHL, your players, their eligibility, it expired. Here you draft a player and you might have four or five years before he’s contributing. Or more. It’s very rarely an overnight thing.
“We’ve seen this before,” he added of the Leafs’ struggles. “It’s part of the process. It takes time. It doesn’t happen overnight. When you’re using analytics, people are [insulting] you and they’re saying, ‘Well, I thought this was supposed to change everything? What about your numbers?’ “It’s not magic. It’s really not magic. It’s a process, and it’s hard work, and it’s difficult, and you have to push your way through it.”Report Typo/Error