Blog

An apple

It has been seven years since Alejandro G. Iñárritu and Emmanuel Lubezki presented CARNE and ARENA at the Cannes Film Festival. In an airport hangar 20 minutes down the coast from the Palais, the pair had created a huge sand-floored volume as participants strapped into virtual reality headsets and found themselves joining a group of immigrants on their perilous crossing from Mexico to the United States . At the time, Deadline called the installation — which will move to LACMA, among other venues — the first step in the birth of an art form.

In the years since, there have been very few developments in virtual reality to suggest that artists have really tapped into the potential of perhaps the most literally immersive form of storytelling imaginable. During the pandemic, Valve’s software sparked VR headset launches — including its own Valve Index — with the release of Half-Life: Alyx, a video game that took huge steps forward for the medium. Tech companies including Meta and Sony have gone all-in on VR, or its hybrid virtual/real cousin, augmented reality. But there are only sparse examples of VR innovation as a medium for artists.

With February’s release of the Vision Pro, Apple, the tech giant known for squared off on nascent technologies that haven’t yet hit the mark, hopes it’s created a platform that might just change all that. Could Vision Pro be the paradigm shift that makes VR and AR an important tool for artists and audiences? The possibilities are tempting.

Even tech reviewers who have seen dozens of headsets come and go have been impressed by the smart implementation of various technologies in the Vision Pro. From its intuitive user interface built around simple glances and gestures, to its lightning-fast response time and high-resolution screens, the developers behind Vision Pro have worked hard to eliminate the many VR pain points that cause an “uncanny valley” effect. ‘, which many users struggle to shake.

Apple’s unique approach to Vision Pro is to try to define a new market segment. The Vision Pro is not a headset that requires any kind of connection to a Mac or PC, nor is it a low-power remote. In fact, it has the same computing hardware as its own top-echelon MacBook Pro built right in. Its visionOS is adapted from the other, highly successful operating systems for its portable devices. Vision Pro is not exclusively a streaming service, PC or video game platform. Just as the iPhone was introduced as an iPod, a telephone and an Internet communicator at the same time, the Vision Pro is designed to serve all these functions simultaneously.

The device has already caught the attention of filmmakers such as Jon Favreau, who created some of the headset’s demo films, and James Cameron, who said his experience was “religious.” But it’s also priced at $3,500 in its most modest configuration, and Apple analysts suggest sales have been sluggish. Many tech reviewers have warned that the Vision Pro, in its current form, may not be the device that lives up to its potential, but that future iterations of the headset may get there. Vision Pro, they say, has to be seen to be believed.

So what might the potential be for the creative industries?

Streaming

Perhaps the least revolutionary of the Apple Vision Pro toolset is its ability as a streaming device for traditional 2D and 3D content. There are already native apps for streamers like Apple TV+ and Disney+, while Netflix and others are available through the web browser.

It’s long been possible to access streaming content through devices like Meta’s Quest headset, whose latest version starts at $499, and Sony’s PlayStation VR2, which retails for $549. But the Vision Pro’s 23-megapixel micro-OLED display technology and improved computing power make streaming a much more enjoyable experience. In keeping with its augmented reality feature, where the device’s two screens – one for each eye – give users a view through the headset and into their own 3D living rooms, throwing a cinema-sized projection onto the wall and watching the ambient lighting from feature films and TV shows dancing around the environment certainly makes the Vision Pro feel more immersive than the competition.

The Vision Pro also improves on Apple’s implementation of Spatial Audio, with two tiny speakers that hover over the user’s ears, delivering surprisingly powerful and effective virtualized 3D audio.

But the device weighs more than 600 grams (1.3 pounds) without its external battery pack, and that battery only offers 2-2.5 hours of semi-charged power, meaning the Vision Pro may still not be a dedicated person’s dream .

Making a movie

With stereoscopic cameras built into the Vision Pro, as well as the company’s latest iterations of the iPhone, the 15 Pro and 15 Pro Max, users can capture awe-inspiring 3D home videos and photos. Enterprising filmmakers will almost certainly be able to take advantage of these technologies for their own storytelling, and while the videos are still framed — meaning they won’t provide a full 360-degree virtual experience — the hologram effect is breathtaking.

The real excitement for professionals, however, comes with Apple Immersive Video. The company created its own camera to capture 180-degree 8K video content. One of the sample clips features a train moving down a snowy, mountainous track, and it seems no accident that the sense of immediate immersion recalls the apocryphal stories of audiences fleeing the theater when they first experienced The Lumière Brothers L’arrivée d’un train en gare de La Ciotat in 1895

The Immersive Video camera is currently an Apple exclusive, but from a seated position its potential is immediate. Want to experience a game without leaving the house? Apple Vision Pro can offer the best available venue on demand and the stage is no longer limited to a theater.

Many VR users report motion sickness when watching 3D immersive content that involves a lot of movement. Apple has carefully selected its sample clips to keep the camera largely steady, or at least minimize motion. Filmmakers will have to figure out how to get footage without making movies that feel like a roller coaster ride.

Live events

Installing Apple Immersive Video at major sporting events and concerts is another attractive prospect. Demonstration footage of Major League Soccer matches with a 3D camera centered just above the goal posts goes one step beyond providing the best seat in the house. Indeed, viewing sports from such a unique perspective provides a whole new perspective on the action.

Another example involves Alicia Keys performing in a rehearsal room. The camera placement once again places the audience in the heart of the action, among Keys and her band. What if Apple Immersive Video came to the Oscars red carpet or the Dolby Theater itself? Suddenly, Hollywood’s hottest ticket might not be as out of reach as it once was.

The real trick for Apple will be keeping the content coming and figuring out the right ways to get it live. With 8K resolution at a high frame rate of 60fps and such a wide field of view, compression will be a key factor. So far, Apple has only demonstrated brief examples of what Immersive Video can do. No current long-form TV shows or movies, nor live events, are available in the format, and early adopters were frustrated by the slow trickle of new footage. Apple will have to move quickly to line up examples to do well in these demos.

production

Perhaps the most intriguing innovation for filmmakers is less about providing an audience. As an augmented reality computing device, using Vision Pro on film sets offers exciting opportunities for all departments.

YouTube filmmakers have already teased Vision Pro’s monitoring capabilities. Video Village takes up valuable space on set, but Vision Pro understands where users are spatially. Cinema-sized screens that monitor camera output can be placed anywhere convenient and will remain there while users move away from them to perform other tasks.

Imagine a costume department where reference images and videos are posted on the walls behind the designers for easy reference as they work. A workspace designed for script supervisors to keep journals with little more than a chair to work from. Focus pullers may no longer require small reference monitors to maintain sharp photos. The use cases are numerous.

Filmmaking is a collaborative tool, and the Vision Pro’s front lenticular screen, which provides a virtualized image of the user’s eye movements from the front of the headset, means the device doesn’t need to be removed for conversations to continue unimpeded . The effect isn’t as pronounced as it appeared in Apple’s early teaser videos, with the device’s reflective front making it more subtle than the company might have hoped, but it’s a clever idea set for future improvement.

Battery life will also hinder the potential use of the kit, especially in remote locations. But could a cache of spare batteries take up less space in an equipment truck than the many monitors, laptops and other devices that are now essential on set?

Apple’s gamble with Vision Pro is a bold one: to carve out a new niche for a technology that had all the hallmarks of near-obsolescence, like 3D cinema before it, when Apple first announced it. Despite a handful of promising VR and AR implementations, Mark Zuckerberg’s vision of the future in the metaverse still feels more like a meme than meaningful, and even the heavy-duty Vision Pro, as beautiful as it looks, struggles to overcome the fatigue of long-term work on handset.

But the first iPhone launched in 2007 with 2G internet and no App Store; just the suite of apps that Apple has designed for it. By the time its successor arrived 18 months later, everything had changed and the smartphone era had truly begun. These technical reviewers are right to envision a fruitful second or third iteration of this concept. And if any company can sink the dollars needed to lower the device and its price, it’s Apple.

However, its success will rely on third parties; filmmakers and consumers willing to embrace its potential. Just as cinema was defined by both its practitioners and those who developed its technology, so the wider industry and audience will be the ones charged with establishing the language of this new paradigm.

The Apple Vision Pro looks like the first device of its kind, making this journey truly worthwhile. Competitors will no doubt emerge to emulate some of this innovation now that the Vision Pro is out in the wild, but even at a price closer to $500, its rivals struggle to overcome the gimmicky nature of VR and the lack of truly revolutionary content to garner high volumes of sales. The Vision Pro is certainly more impressive than its competitors technologically. Only time will tell if Apple’s expensive roll of the dice will really ignite the future of a new artistic medium.