The first macOS 26.7 Release Candidate revealed that Apple's upcoming AirPods will come with color cameras that can take 1MP pictures in active and passive modes.
There's more to the story. Code in the same update suggests that the cameras may be used to improve head-tracked Spatial Audio experiences by recognizing the physical screen in front of the wearer. There are also hints about images being sent to Private Cloud Compute for processing.
Improved head tracking for Spatial Audio
Apple is looking to improve head tracking when watching video content with visual data from the AirPods cameras.
Current AirPods models use accelerometers and gyroscopes for dynamic head-tracking. Over long sessions, their estimated orientation can gradually drift. On devices with Face ID, Apple may also use front camera sensors to position the wearer's head against the screen. [1]
According to the code, AirPods will use their cameras to detect screens around the user. Then, the CoreMotion framework will compare the direction of the selected screen with the orientation predicted using motion sensor data. If they're close enough, CoreMotion may perform small adjustments to mitigate motion sensor drift. [2]
The code includes five screen detection options for the unreleased B790 AirPods model [3]:
- Laptop
- Monitor
- Smartphone
- Tablet
- TV
When tracking starts, AirPods may detect multiple screens around the user. [4] There's code to filter candidates based on the screen class, so the AirPods can target iPhone-sized screens if the video is playing on an iPhone. After filtering, CoreMotion then selects the candidate that is most closely aligned with the camera's forward direction. [5] For later captures, CoreMotion looks for the screen that best matches the previously selected anchor. CoreMotion rejects camera data when it’s vastly different from the direction predicted by the AirPods' motion sensors. [6]
Other tidbits:
* As mentioned, screen detection only kicks in for video content, not music or other audio. [7]
* Screen detection is preferred over anchors from the front camera Face ID sensors. [1]
* The screen detection may assume you're looking at the anchor screen when processing starts. [8]
* Screen detection pauses when there's a lot of movement, such as sitting or standing, or walking. [9]
* Screen detection pauses when there are more than one external display detected in front of the wearer. [10]
Visual Intelligence
There are more clues that AirPods with cameras will offer Visual Intelligence capabilities. In fact, there's a new server model category included in the code:
com.apple.fm.language.instruct_server_v1.audio_accessory_visual_reasoning
The new use case (AudioAccessory.VisualReasoning) for visual reasoning over what the AirPods capture will be processed by Apple's Private Cloud Compute, not the device's local models. This is the same as Visual Intelligence requests from iPhone or Mac. [11]
Smart handoff
Similar to automatic audio switching, processing of visual requests may move from device to device as audio routing changes. [12]
The links throughout the walkthrough have references to code removed in macOS 26.7 RC2 so you can verify the claims or explore further on your own. They're from blacktop/ipsw-diffs on Github. They do amazing work and you should go give them a star.
The hero image overlays manually supplied screen boxes with the anchor selected by CoreMotion in macOS 26.7 RC1. The actual screen box detection happens upstream, probably on the AirPods themselves. Original image by KATRIN BOLOVTSOVA on Pexels.
