As expected, Apple announced its mixed reality mask during its WWDC 2023 conference. If the concentrate of embedded technologies is enough to make your head spin, the uses offered raise real questions about the future of computing.
As planned for months, Apple took advantage of its developer conference, the famous WWDC, to enter the world of virtual reality. No less than 45 minutes were reserved at the end of the Keynote for one of those famous “One More Thing” (one more thing) dear to Apple to present the Vision Pro, the mixed reality mask signed by Apple. Better to speak of a goggle actually, rather than of a helmet, the Vision Pro looks so much like, as expected, a large ski goggle, with a single headband passing behind the head, very wide and very comfortable in appearance, even if it will have to be checked in use of course. No need therefore for a support band passing over the head: the mask is light enough. Normal, since unlike the MetaQuest headsets from Meta/Facebook for example, the Apple Vision Pro does not carry a battery: it is used either plugged into the socket, or plugged into its small external battery that is kept in his pocket. In the end, the Vision Pro presented on June 5 corresponds in almost every detail to everything that Bloomberg journalist Mark Gurman had revealed to us, over the months leading up to the official announcement.
Vision Pro: the first steps in spatial computing
More than virtual reality, we are indeed talking about mixed reality, as explained very well by Mark Gurman, guest of the podcast iWeek Apple Week (episode 135 of may 4, 2023): “ The two main realities are virtual reality and augmented reality. Virtual reality is the one in which we are completely locked up. These are screens only. Augmented reality is what we see through transparent glasses, a bit like Google Glass. With information projected superimposed on what we see naturally. Mixed reality is primarily VR at its core, but with an integrated augmented reality mode. I call it false augmented reality because instead of using your eyes directly, through transparent glasses, you use cameras, which reproduce what is in front of you and this is what characterizes mixed reality , this ability to switch from VR to AR, at any time ». This mixed reality mode is controlled using a wheel with a design almost identical to that of the digital crown of the Apple Watch. Located on the right of the upper edge of the mask, this wheel therefore makes it possible either to isolate oneself completely from the outside world, or to display inside the helmet, therefore in front of the eyes of its user, the image captured by the external cameras of the helmet. By pushing the knob all the way in, it’s as if the mask became transparent.
Indeed, at the base of the front face of the helmet, the one facing outwards therefore, cameras constantly film what is happening in front of you and can display on the screens in front of your eyes the room in which you are: you can so you can move around without fear of bumping into furniture, for example.
These cameras are supplemented by a lidar and depth sensors, which allow the mask to calculate with precision all the objects, furniture and walls that surround you, to superimpose the virtual objects displayed by the screens that you have in front of your eyes.
Thus, when you display the window of an application for example (see below), it will seem to float in the air above your coffee table for example, by casting a shadow on the latter: virtual objects interact with objects real in the room.
23 million pixels in front of the eyes
Inside the mask, two micro-OLED screens are placed naturally in front of your eyes. Despite their reduced surface, each screen concentrates more pixels than in a 4K screen, whose resolution is 3840 x 2160, i.e. a total of just over 8 million pixels! This is, according to Apple, the minimum to ensure sufficient quality in particular so that the pixels are no longer discernible at this distance and allow comfortable reading of text. For this comparison, Apple indicates that in the area occupied by a pixel on an iPhone, the mask in place 64! This goes far beyond what all other competitor VR headsets offer today. For glasses wearers, Apple has called on the famous brand Zeiss for optical inserts that are added magnetically to the inside of the mask to adapt the screens to your sight.
Vision Pro: what functions?
As in other areas, that of smartphones for example, Apple is not the first to present a VR / AR headset. Meta, Facebook’s parent company, has been offering its Meta Quest headsets for several years, allowing visitors to visit its metaverse, that is to say its virtual 3D universe, but, it must be said, without really convincing. And that’s why Apple spent a lot of time during the Keynote to describe multiple use cases. Enough to convince? Only the future will tell us.
Upon release early next year (see below), the Vision Pro will be compatible with hundreds of thousands of apps straight from Apple’s usual App Store. Normal finally, since thanks to its integrated M2 processor, the mask can directly run almost all the iPhone and iPad applications whose windows you place all around you. You are no longer limited by the physical screen you have in front of you, or even the second screen connected to your MacBook: you arrange the windows of the apps as you see fit in the space.
Among the other uses mentioned by Apple, there are also FaceTime conversations. It is easy to understand that you will see the images of your correspondent(s) displayed directly in front of your eyes. But what will they see? There are indeed cameras inside the helmet directed towards your eyes, but they are not used to reproduce your face, it would be impossible. During the Keynote, Apple briefly showed that when you first start the headset you will be asked to take the headset in your hands and place the front of it in front of you. In this way, the cameras and depth sensors will be able to create a 3D representation of your face. And it is this avatar of yourself that will be sent to your correspondents. Lively of course. We can’t wait to see what happens and in particular to see if the result is convincing.
Note that it is this same 3D representation of the part of your face hidden by the mask that will be reproduced on the screen located on the front of the mask when you want to interact with a physical person who is in the same room as you. Thanks to the outside cameras, you will see it. And the person in question will also be able to speak to you naturally without having to address a “blind” mask. It might be weird though, especially at first.
And to rest from all these efforts, all you have to do is flop down on your sofa to enjoy a movie session as if you were there where, again, the helmet will be able to simulate a huge screen of cinema in front of you. And you can enjoy the spatialized sound broadcast by the small speakers integrated into the headband. With all the same a big difference with a real movie theater, it is that you will be there all alone.
Which processor for the Vision Pro?
So you can go anywhere with the headset, it’s designed to be self-contained, meaning it doesn’t need to be wired to a computer or smartphone to work. It embeds all the computing power necessary to manage the display and all the 3D calculations necessary to immerse you in a world in virtual or mixed reality. The Vision Pro relies for this on two processors. First of all, an M2, identical to that found in Macs or iPads, in charge of running applications. And a brand new processor called R1 responsible for coordinating everything in order to minimize latency between the images collected by the cameras and their display in front of your eyes. Apple has indicated that this latency will be around 12 ms maximum, short enough to avoid the nauseating effects that can easily be felt when using a VR headset.
Vision Pro: what autonomy?
As we saw above, the Vision Pro is also capable of operating for two hours on an external battery, which you place in your pocket. Two hours may seem short, especially if you want to use the headset to watch a film that would make three. In this case, you can still connect it to the mains, which will not pose a problem since you are often seated without moving when watching a film. There remains the problem of watching a long movie while traveling by plane.
As expected, Apple’s VR/AR mask comes without a controller. In other words, all manipulations of objects or menus that we will have in front of our eyes will be done directly with the hands, without going through an accessory. External cameras pointing downwards are therefore responsible for precisely capturing the movements of your hands and fingers to allow you to grab a virtual object, type on a virtual keyboard or simply point and press on an element of menu. The approach would therefore be very different from that proposed by the Meta Quest which comes with two joysticks for all manipulations.
As mentioned above, cameras located inside the mask constantly film your eyes. Thanks to them, you can perform a number of manipulations without hands, simply by looking at a specific place. For example, you will only need to look at an input box to start dictating your search keywords. You can also scroll through a page eyes, simply with your eyes.
Vision Pro: what release date?
Don’t order it from Santa, Apple’s Vision Pro mask won’t be available until early 2024, with no specific date. What’s more, it will only be offered on the American market, other countries will be added to the list… later. Obviously, Apple does not yet have a very clear vision of this brand new market or, above all, of its manufacturing capacities. It must be said that the technologies used are state-of-the-art. According to an analyst from Morgan Stanley, production would not start until the end of next October and would not reach mass production until December. And, despite this, we will surely be very far from the tens of millions of iPhones produced each quarter.
Vision Pro: what price?
This is one of the rare details on which the rumors were all wrong. While we expected a price of around 3000 dollars, it is actually 3500 dollars that you will have to pay to afford the Vision Pro! And let’s not forget that US prices are always advertised without tax, since the added tax is different in each state. In other words, we can expect a price of at least 4200 euros including tax in France! And as it is also necessary to anticipate the vagaries of exchange rates, it is a safe bet that the price will be rather 4499 euros including tax. Decidedly, the VR / AR mask from Apple will indeed be reserved for professionals at first. And we will have to wait until at least 2025, if not 2026, to hope to see a slightly cheaper non-Pro version.