Skip to main content

At the 2017 U.S. Open in New York, IBM’s Watson extracted key moments from the 26,000 points scored by tennis stars such as Canadian Denis Shapovalov.

Corinne Dubreuil/Abaca Press

If there's one thing sports fans take for granted, it's highlight reels. After all, just about no one who enjoys watching their team's best plays is liable to think about the video editors who spend countless hours assembling such clips.

Fortunately for those editors, artificial intelligence might make their thankless jobs a little easier.

"It really allows the editorial and video production team to focus on creating more custom highlight packages," says Stephen Hammer, chief technology officer for IBM's sports and entertainment division. "They don't have to spend any time at all on the highly repetitive video-editing task associated with the match summary highlights."

Story continues below advertisement

IBM is actively deploying its artificial intelligence video highlight editor after trying it out at the U.S. Open tennis tournament in New York this past summer.

Using Watson, the super-computer that famously beat its human opponents on the game show Jeopardy! in 2011, the system can scan dozens of hours of footage and pick out the most pertinent moments for inclusion in highlight packages.

At the U.S. Open, Watson looked at more 26,000 points scored. In many cases, it was creating reels within two minutes of a match ending, then posting them to Facebook and U.S. Open apps.

"For a video editor to go through and watch that many points and pick out exciting moments, that would be a huge task," Mr. Hammer says. "It really helped them focus on other things."

To achieve its speedy results, Watson first had to be trained. Before turning to tennis, the IBM team created a proof-of-concept using footage from the Masters golf tournament this year.

They taught the computer to recognize certain elements of the game, including crowd noise, excitement from commentators and body language from players – fist pumps and high-fiving caddies, especially.

Any of those marked the particular moment as important to Watson. With a full complement of highlights, the computer then ranked them against each other to come up with a distilled set of the best. As with all AI, the more data the machine digests, the more accurate its results.

Story continues below advertisement

IBM then trained the system on footage from the Wimbledon tennis tournament in July before launching its Watson Media business and partnering with the U.S. Tennis Association for the U.S. Open.

Watson Media isn't just aiming for further deployments in sports, however – the broader entertainment field is a target, too, owing to the system's origins.

IBM first applied AI to highlights in the form of a trailer for Morgan, a horror movie released last year. Fittingly, the movie is about an artificially intelligent entity.

As with its sports training, Watson studied footage of previous horror movies to learn how to identify key moments. Screams, tense music and what Mr. Hammer calls "highly emotional moments" eventually made the cut. Further movie trailers are possible, he adds.

IBM is far from the only technology company applying AI to video sorting and editing.

Adobe, the maker of graphics editing program Photoshop, this summer announced a collaboration with Stanford University on a program that automatically studies footage and suggests edits. The system can be taught the user's editing style and, like Watson, promises to save copious hours of grunt work.

Story continues below advertisement

Also this summer, Google announced it is using AI to scrub extremist content from YouTube. The company says the new system is more efficient than humans at the task, removing hate-filled videos at double the rate.

"Over 75 per cent of the videos we've removed for violent extremism over the past month were taken down before receiving a single human flag," a YouTube spokesperson said in a blog post.

Media experts say such efforts are just the beginning. The application of AI to highlights and trailers is a natural and logical step because it will free up human editors from the tedious work of poring over footage.

Their jobs are likely to be elevated into more of an oversight role as a result.

"Humans still have to put the coherence to it, they have to give it a flow and tell a story," says Robert Clapperton, assistant professor at Ryerson University's school of professional communication in Toronto.

"AI is going to be a team member within a group of human beings that does specific work. Maybe there are three editors that put together a sports highlights show and one of them is now going to be AI whose job is to do the rote stuff."

Report an error Editorial code of conduct
Tickers mentioned in this story
Unchecking box will stop auto data updates
Comments

Welcome to The Globe and Mail’s comment community. This is a space where subscribers can engage with each other and Globe staff. Non-subscribers can read and sort comments but will not be able to engage with them in any way. Click here to subscribe.

If you would like to write a letter to the editor, please forward it to letters@globeandmail.com. Readers can also interact with The Globe on Facebook and Twitter .

Welcome to The Globe and Mail’s comment community. This is a space where subscribers can engage with each other and Globe staff. Non-subscribers can read and sort comments but will not be able to engage with them in any way. Click here to subscribe.

If you would like to write a letter to the editor, please forward it to letters@globeandmail.com. Readers can also interact with The Globe on Facebook and Twitter .

Welcome to The Globe and Mail’s comment community. This is a space where subscribers can engage with each other and Globe staff.

We aim to create a safe and valuable space for discussion and debate. That means:

  • Treat others as you wish to be treated
  • Criticize ideas, not people
  • Stay on topic
  • Avoid the use of toxic and offensive language
  • Flag bad behaviour

Comments that violate our community guidelines will be removed.

If your comment doesn't appear immediately it has been sent to a member of our moderation team for review

Read our community guidelines here

Discussion loading…

Due to technical reasons, we have temporarily removed commenting from our articles. We hope to have this fixed soon. Thank you for your patience. If you are looking to give feedback on our new site, please send it along to feedback@globeandmail.com. If you want to write a letter to the editor, please forward to letters@globeandmail.com.