How to control Apple Vision Pro? Gestures, voice, keyboard, gaze…

• The Apple Vision Pro mixed reality headset can be controlled by voice and Bluetooth keyboards
• Mouse, Magic Trackpad and controllers are also compatible
• Alternatively, gestures and eye tracking allow you to move around and interact with the operating system.

Apple unveiled the mixed reality headsetVision ProMonday, but we were only able to see the device in real conditions very quickly during the event. For you, we have therefore taken the time to analyze the documentation dedicated to developers and published in the days that followed until today, the closing date of the Worldwide Developer Conference.

In this guide, you will find the main solutions for navigating throughvisionOS, the new operating system that equips the Vision Pro. Many apps already present on Mac are included, such as Notes, Apple Music,FaceTime, Freeform or even Mail and Mindfulness. So here's how to control them.

With your eyes

First of all, you need to know how to move around within an app. To do this, nothing could be simpler: the helmet, thanks to multiple integrated sensors and patented technology,follows your gazeand knows exactly where it lands. Marques Brownlee, who tested the Vision Pro in Cupertino, describes this very precise system as “telepathic“. The headset thus guesses what you are viewing, and you do not have to do anything additional to tell it. This solution replaces your mouse pointer.

In addition, when your gaze stops on an icon, it bounces, without the slightest latency. Infrared cameras, projecting invisible light onto the retina, enable this feat. To see what the result looks like, here is a demonstration:

With gestures

Once your eyes land on an element you want to interact with, the Vision Pro is then controlled with your hands. Apple reminding us that this solution is the most natural since we already use it for most daily tasks. Here, there are several gestures to be aware of. All must be carried out withthumb and index finger together, like a pinch. No matter your position (standing, sitting, etc.): the Vision Pro detects your fingers.

One-handed

• a swipe gesture from bottom to top scrolls content down, for example to read a newspaper article
• pinch then move corresponds to drag / drop
• tapping the index finger against the thumb is equivalent to a single click with the mouse
• tapping twice is equivalent to double clicking
• just pinch for several seconds to trigger a long press, for example to display shortcuts for an app, like on iPhone

With both hands

• by moving both hands away, you can zoom
• by bringing them closer, we zoom out
• as below, it is also possible to rotate by turning your hands clockwise:

Siri is here

Unsurprisingly, Apple has also integrated its voice assistant Siri into the Vision Pro. To call him, all you have to do is pronounce his name. There is no longer any need to add “dis” just before. And if you want to write an email or a spoken message, it's also possible with thedictation.

Compatible accessories

Although the Vision Pro comes with a virtual keyboard, Apple admits that it remains impractical when it comes to typing in the air (in a vacuum) for an extended period of time. This is why the headset is compatible withMagic Keyboard, Apple trackpads and mice. And since Bluetooth is available, we assume that third-party accessories will also be supported. This is already confirmed for the controllers, even if the name of the first partner manufacturers has not yet been revealed. ThereDualSensefrom Sony could however be seen in the presentation videos.

And theDigital Crown, in all this?

Finally, let us discuss the case ofDigital Crown. Already present on the AirPods Max and the Apple Watch Series 8 and Apple Watch Ultra, this wheel is used on the Vision Pro toadjust the intensity of virtual reality. Thus, by rotating it, you can choose to completely immerse yourself in a 360-degree setting, while in the other direction it is, on the contrary, the user's real environment which is displayed in transparency. The apps then remain visible, superimposed on the outside in augmented reality.

Note: a second dial is also on the right side, on the headband which allows the helmet to be held in position on the face. This is simply used to adjust the tightness against the skull.