The story seems, at the distance of an arrogant century, to be almost laughable: 120 years ago this month, in Lyon, France, a handful of people watching a 50-second silent film by Auguste and Louis Lumière were sufficiently terrified by the flickering black-and-white moving image of an approaching locomotive that they leapt out of their seats. How naive! Surely we, the savvy species that can stream the entire Lumiere Bros.’ catalogue during our morning commute aboard sleek modern trains, cannot be so easily moved.
And yet, on a sunny day last winter in a downtown Toronto office, I strapped on some headphones and a simple virtual reality headset – really, just a pair of special goggles attached to a smartphone – and suddenly felt as if I was amid the ice floes of the Northwest Passage; as night fell, the aurora borealis fluttered above my head like a holy green ghost. Some months later, wearing a higher-tech headset, I found myself underwater, perched on the prow of a sunken wooden boat, my mouth agape as a blue whale swam up, eyed me ominously, then floated away. It was as heart-stopping and real as an approaching locomotive.
Virtual reality is frequently compared to teleportation, or the Star Trek holodeck: You are instantly transported to a different place, a different time, a different environment, a different consciousness.
My apologies; I realize that sounds absurd. There is a terrible tendency, in writing or talking about VR, to sound like a carnival barker. In a TED Talk, the filmmaker Chris Milk, who showed his VR documentary of a Syrian refugee family, Clouds Over Sidra, to attendees of the 2015 World Economic Forum, said of virtual reality: “It’s a machine. But through this machine we become more compassionate, we become more empathetic, and we become more connected. And ultimately, we become more human.”
Felix & Paul Studios
“There are two groups when it comes to VR,” offered Bernie Roehl, a Waterloo-based developer, in a speech during the Immersed VR conference last fall at the Ontario Science Centre. “There are people, like I am, who are absolutely convinced that VR is the future – not only of computer gaming, but of all entertainment and online social interaction. And there’s another group – who haven’t tried it yet. It is that powerful.”
It is also coming very soon to a home near you, with the potential to transform not just gaming, entertainment and social media but social justice advocacy, tourism, education, marketing, shopping, and dozens of other everyday experiences. Last Wednesday the Oculus Rift, a virtual reality rig produced by a four-year-old startup which Facebook bought in 2014 for $2-billion, went on sale in 20 countries: the first deliveries are due in March. VR companies were so prevalent at this week’s manic Consumer Electronics Show in Las Vegas that the online news outlet CNET produced a report which owners of VR gear could watch at home in, yes, virtual reality. Later this month, the Sundance Film Festival will feature 30 VR experiences, including two documentary shorts from a Montreal-based studio that is an early leader in the field, as well as 20th Century Fox’s The Martian Experience, a 20-minute interactive adventure which places viewers in the same predicament as Matt Damon’s marooned astronaut in that studio’s recent blockbuster film.
But while many fans believe that, after decades of failed promises, VR’s time has finally come – various industry estimates project the market could be worth between $70-$150-billion (U.S.) by 2020 – many creators are wary of the hype. They speak of a gold-rush-like frenzy gripping the industry: of unrealistic expectations, of devices priced for The One per cent, of a flood of inferior films and other experiences that might quickly turn off potential users. Some critics warn that VR documentaries, one of the platform’s especially popular genres, travel an ethical minefield studded with the potential for fakery, manipulation, and a discomfiting treatment of their subjects.
And, and also? If you think Facebook and Google know a lot about your life, just wait until you start interacting with the world through VR, which will soon be able to monitor your every eye movement, curl of the lip, and involuntary finger twitch.
The Canadian leaders
The dream of immersive cinema, of a film experience that could duplicate reality, has existed for more than half a century. In the 1950s, Cinerama took audiences on a roller coaster ride with 146-degree vision; IMAX rolled out in 1971 with a vertigo-inducing flyover of Lake Superior. Planetariums give viewers the sense that they are gazing up at the real night sky.
But each of those cases involves an image projected onto a framed screen (or a domed roof); the viewer watches the scene, rather than feeling as if they are inside of a three-dimensional, 360-degree environment.
With VR, “for the first time, you have a medium that breaks the fourth wall,” suggests filmmaker Thomas Wallner, whose Toronto-based company DEEP Inc. produced a 360-degree video companion piece to last year’s TVO series The Polar Sea. (Proper virtual reality technology, which tracks every minor head movement – without which nausea can ensue – requires high-powered computers; a user does not have the same power to move through a space in mere 3D 360-degree video.) “It eliminates the distance between the observer and the observed in a way that has never happened before in the history of media.”
After decades of fits and starts in technological development, virtual reality suddenly began to feel very real in 2012, when Oculus VR, a start-up founded by the then-20-year-old prodigy Palmer Luckey, launched a Kickstarter campaign to raise funds for what it was calling “the first truly immersive virtual reality headset for video games.”
Two years later, the company was snapped up by Facebook, in part because the social media giant believes VR will become a common communication platform: People in different locations will game together in the same virtual space (as they currently do in non-VR online gaming environments). They will chat or shop, watch movies in a shared virtual living room, or maybe, instead of Skype, they’ll drop in – virtually – on a birthday party from across the country to watch grandma blowing out the candles.
Some people are raising red flags over that possibility. In a recent TEDx Talk, Ana Serrano, the director of the Media Lab at the Canadian Film Centre, suggested that massive software engines might be able to read physical cues – subtle eye movements, turns of the head – to discern when we are at our most susceptible to commercial persuasion. Facebook and Google, after all, make their money by transforming our behaviour into data for marketers.
Felix & Paul Studios
“The Internet has become an economic engine devoted to converting big data from billions of people into large profits for a small handful of players,” she noted.
Until that social capability becomes a reality some years from now, creators are rushing in to stake a virtual claim in a new gold rush, drawn by the belief that the field is so new that few have a head start, and anybody could wind up as the Louis B. Mayer of cinematic VR.
It won’t be that easy.
“We’ve seen a lot of experiences being created in the past few years that were very opportunistic, and didn’t really make an effort of trying to convey an artistic voice or vision for VR,” said Félix Lajeunesse, co-founder of the Montreal-based studio Felix & Paul, which has carved out a fast niche in VR. “I don’t think this is helping VR in any sort of way.”
“VR is easy to mess up and it’s hard to make it work,” he added. “It’s a real craft, just like cinema, just like painting, just like music. It’s not magical.”
Since jumping into VR in late 2013, Lajeunesse says Felix & Paul has scaled up quickly, from three to 30 staff. The company created its filmmaking process from scratch, building a 360-degree camera rig and all of the postproduction software for the tricky tasks of stitching together the footage, adding binaural sound, and making sure the experience doesn’t cause nausea. With U.S. clients including Oculus and 20th-Century Fox, they’ll be opening an office in Los Angeles this year.
Their first piece, Strangers, was a quiet number which placed the viewer in the Montreal studio of the musician Patrick Watson as he sits at the piano and works up a song. “The proposition of VR is pretty powerful, and a lot of the stuff we’re focused on is not so much the spectacle coming out of VR, but the intimacy of the medium,” notes Lajeunesse.
“ Strangers was a one-on-one experience with a musician in a studio. It’s really about you as a viewer, and him as an artist sharing a space, and being together in a moment of what you would call vulnerability, because he’s not peforming, he’s writing music.” The moment felt so real to some viewers that, when they turned away from Watson to gaze around the room, they said they felt as if they were being rude to him.
The craft on display in Strangers was part of what prompted Fox to commission a short piece by Felix & Paul based on the drama Wild, called The Wild Experience, which placed the viewer in an intimate moment in the woods with the mother and daughter from that film, played by Laura Dern and Reese Witherspoon, respectively.
Around the same time, Felix & Paul signed a deal with Oculus to create a series of experiences for Rift users – The Hollywood Reporter dubbed the pair “Virtual Reality’s Wannabe Spielbergs” – which includes this month’s Lebron James: Striving for Greatness, in which the viewer feels like a member of the basketball star’s entourage, watching him train for the current season.
Later this month, Felix & Paul will bring two VR documentaries to Sundance: Nomads: Masai and Nomads: Sea Gypsies, companion pieces to last year’s Nomads: Herders, which dropped viewers in among nomadic yak herders in Mongolia.
The National Film Board of Canada will also be bringing two VR pieces to Sundance, including The Unknown Photographer, a so-called “documentary hybrid” built up from a collection of World War I photographs discovered in a Quebec farmhouse in 1974. (The piece was co-produced by the Montreal studio Turbulent.) Producer Louis-Richard Tremblay says viewers begin the experience feeling as if they are in a war re-enactment, walking through trenches; the photographs are monumental, life-sized. Slowly, the experience becomes more dreamlike: you feel as if you are flying.
“The goal is to talk about war from a perspective that only an immersive experience could provide,” said Tremblay. “It has that fluidity, openness, it relates to how we think the brain works.” He added: “It was a challenge in so many ways, just because the technology is so new, we’re doing it for the first time, there’s no one you can call.”
The VR creation process at the Toronto-based marketing services firm Secret Location is proudly one of trial and error. “With filmmaking and radio and text, you have generations of work to be able to build off of,” explains president James Milward. “Making content in this form needs to be a lot more iterative.”
“It really is a fundamentally new cinematic language.”
Last September, Secret Location won a Creative Arts Emmy Award (in the best user experience and visual design category) for a virtual reality piece it did for the TV show Sleepy Hollow, marking the first time a VR experience had won an Emmy. Though the company is primarily focused on client work, it is starting to develop its own intellectual property: it is currently in the early stages of adapting Stephen King’s novel Insomnia as an episodic series of VR experiences.
While Milward acknowledges Insomnia could be adapted for traditional film or TV, he says the use of VR will make the experience richer. Still, he acknowledges, you don’t want to use VR gratuitously. “The question becomes: What’s the point of the immersion?”
Felix & Paul Studios
In search of empathy
Later this month, three VR experiences Secret Location produced for the publisher Pearson and the not-for-profit Project Literary, exploring the experience of illiteracy, will make an appearance at the World Economic Forum. Like Chris Milk’s Clouds Above Sidra, the hope is to evoke empathy among Davos attendees for those on the margins.
This is a common theme among some of the early VR documentaries. But, while there are high hopes for the technology – in his TED Talk last April, Chris Milk said VR “connects humans to other humans in a profound way that I’ve never seen before, in any other media” – some creators say there are some dangerous ethical and aesthetic pitfalls.
Last November, the New York Times posted an 11-minute 360-degree film of three children displaced from their homes in Ukraine, Lebanon, and South Sudan, and distributed one million free pairs of Google Cardboard, a rudimentary VR headset designed for use with Apple or Android smartphones.
The paper’s public editor wrote a column noting that a few critics were concerned that the cumbersome VR-filming gear can require someone to direct subjects – a no-no in classical documentary filmmaking. Others felt the film was voyeuristic, a common criticism of VR documentaries, which often aim to simply facilitate a viewer’s so-called ‘presence’ in a scene.
“What’s interesting is, you’ve had hundreds of years of documentary aesthetic and filmmaking theory being developed with linear films, and the moment 3D 360 emerged, many of the VR filmmakers have forgotten some of the rules and conventions of what documentary filmmaking could be,” said the CFC’s Ana Serrano.
“So, you have lots of observational, voyeuristic types, with the 3D 360 filmmaking being touted as awesome docs, whereas if they were shown on linear screens, they would be panned.”
At the moment, filmmakers are banking on the hope that merely offering viewers ‘presence’ in a scene will be enough to produce empathy. In an earlier era, photographs themselves did the trick, showing people the manufactured reality of another person’s existence. In time, that medium lost much of its power. Will VR suffer the same fate?
“With any medium that becomes more pervasive, we become desensitized,” acknowledges Milward, of Secret Location. “I think that you’re certainly going to have a higher threshold for the experiences that are truly creating empathy. Just as with the best photograph from a National Geographic cover, we can still find that deep sense of empathy. It’s just not every photograph anymore.”
“Right now, we’re still caught in the novelty act of VR,” he adds. Soon, he says, we’ll need stories, or “something you’d demand in any other medium to make it meaningful. So I think that’s inevitable. We’re going to have to continue to push how we make things, and what we choose to make, in order to make it meaningful.”