Apple Outlines How Smart Glasses Could Record Jitter-Free, Gaze-Driven Video

Multiple reports suggest that Apple is developing an augmented reality headset as well as smart glasses, and a newly granted Apple patent may provide a clue about one of the potential features that could be included in the latter device.

patent
The U.S. Patent and Trademark Office on Tuesday granted Apple a patent describing "systems and methods for gaze-driven recording of video."

Apple's patent notes that head-mounted displays are used to provide virtual reality, augmented reality, and/or mixed reality experiences for users, and envisions video from such a device being recorded for later playback or analysis. Specifically, it envisions such a system in smart glasses, where the use of built-in gaze-tracking sensors could provide an indication of where a person is currently looking, which could direct a built-in camera to record the scene where the user's eyes are trained, instead of simply recording what's in front of the user.

The patent envisions a system with one or more gaze-tracking sensors, one or more image sensors, and a processing apparatus configured to access the gaze data captured. This apparatus would apply a temporal filter to the gaze data to obtain a "smoothed gaze estimate," which would allow it to determine a region of interest based on the sensors. The system would then apply signal processing to the video based on the region of interest to obtain an enhanced recording that could mitigate the impact of rapid-movement eye saccades.

According to the patent, the system would consist of a pair of smart glasses and a separate connected device for power and storage, which could be interpreted to mean an iPhone, iPad or Mac.

record gaze tracking patent 2
The invention harks back to another patent filed last month in which an eye tracking system is used for detecting the position and movements of a user's eyes in smart glasses and other head-mounted displays.

In the latter patent, Apple envisages the eye tracking system being used to analyze the position and movements of the user's eyes, or to detect other information such as pupil dilation. In one example, identifying the point of the user's gaze allows for interaction with content shown on the near-eye display of the head-mounted display, while another application includes the creation of eye image animations that could be used for digital avatars visible to other users in a mixed-reality communal environment.

Of course, Apple files numerous patent applications every week, and many of these inventions do not see the light of day. But they do provide a glimpse into the kinds of technologies Apple is exploring for potential use in its upcoming AR/VR products, and based on the rumored launch roadmap for its mixed-reality headset and smart glasses, the company still has plenty of time to implement them.

The Information and Bloomberg have both said that Apple is working on smart glasses AND an AR/VR headset, with the headset to come out in first followed by the glasses. Apple analyst Ming-Chi Kuo believes the "mixed reality" headset will come out in 2022, with the Apple Glasses to follow in 2025. The headset is AR/VR, while Apple's glasses are augmented reality.

The headset is rumored to be similar to Facebook's Oculus Quest virtual reality headset, but with a sleeker design that uses fabrics and lightweight materials to ensure the headset is comfortable. JP Morgan believes the headset will look similar to other brands of VR headsets, featuring six lenses and an optical LiDAR scanner to map the wearer's environment. The headset will be targeted at the top end of the consumer market and will be more expensive than others on the market.

The glasses, meanwhile, which are reportedly in an early stage of development, are said to resemble high-end sunglasses with thick frames that house the battery and chips. Kuo expects the AR glasses to be marketed as an ‌iPhone‌ accessory and will primarily take a display role offloading computing, networking, and positioning to the ‌iPhone‌, with the glasses providing a mobile-first "optical see-through AR experience."

Patently Apple was first to report on the patent being granted today. Apple originally applied for the patent in December 2019.

Related Roundup: Apple Vision Pro
Tag: Patent
Buyer's Guide: Vision Pro (Buy Now)
Related Forum: Apple Vision Pro

Popular Stories

election results 2024 live activities

Track 2024 U.S. Election Results Live on Your iPhone Lock Screen

Tuesday November 5, 2024 5:02 am PST by
Apple News is providing Live Activities support for the 2024 U.S. presidential election, allowing iPhone and iPad users to track electoral results in real time directly from their Lock Screen. The feature is rolling out for U.S. users over the course of Election Day, November 5, providing continuous updates of the electoral count. So if you're interested, you don't need to repeatedly check...
Generic iOS 18

Everything New in iOS 18.2 Beta 2

Monday November 4, 2024 12:34 pm PST by
Apple today seeded the second betas of upcoming iOS 18.2 and iPadOS 18.2 updates to developers, and Apple is continuing to refine the Apple Intelligence capabilities. There are also a handful of smaller features that are worth knowing about. Find My Find My has a new option to Share Item Location with an "airline or trusted person" that can help you locate something that you've misplaced....
iCloud General Feature

Here's What's New in Apple's Updated iCloud Terms and Conditions Taking Effect Next Week

Friday September 13, 2024 7:39 am PDT by
Apple has started notifying users about an upcoming revision to its iCloud Terms and Conditions, which takes effect on Monday, September 16. We compared the text of the upcoming iCloud Terms and Conditions with the current U.S. version from September 18, 2023 and identified four key changes: "Apple ID" references have been changed to "Apple Account" throughout. iCloud users must agree to ...
ipads early bf deals

The Best Early Black Friday iPad Deals

Tuesday November 5, 2024 7:02 am PST by
Black Friday is still a few weeks away, but you can already find great prices on numerous iPads, including the 9th generation iPad, 10th generation iPad, iPad Air, and iPad mini. Note: MacRumors is an affiliate partner with some of these vendors. When you click a link and make a purchase, we may receive a small payment, which helps us keep the site running. Of course, there is a chance that ...
ios 18 2 chatgpt plus

iOS 18.2 Beta 2 Shows Siri ChatGPT Limit, Offers 'Plus' Upgrade Option

Monday November 4, 2024 10:54 am PST by
With the second beta of iOS 18.2 that's available for developers today, Apple has further fleshed out the ChatGPT integration that's available with Siri. In the Settings app, there's now a section that shows the ChatGPT daily limit, and offers an option to upgrade to the paid ChatGPT Plus plan. The beta includes an Advanced Capabilities section with a "Daily Limit" reading that shows up as...
M4 MacBook Pros Thumb

M4 MacBook Pro Reviews: Processor Benchmarks Impress, New Nano-Texture Option Worth the Extra $150

Thursday November 7, 2024 6:14 am PST by
The first wave of reviews of Apple's new M4-powered MacBook Pro models were published this morning. We've collected some of the latest impressions from YouTube channels and select media outlets below. Apple last month announced the new 14-inch and 16-inch MacBook Pro models, adding next-generation M4, M4 Pro, and M4 Max chips, with Thunderbolt 5 ports on higher-end models, display and camera ...
early apple watch black friday

The Best Early Black Friday Apple Watch Deals

Wednesday November 6, 2024 6:33 am PST by
Black Friday is just around the corner, and Apple Watch deals have begun appearing ahead of the shopping holiday on November 29. In this article, we'll take a look at all of the best early Black Friday Apple Watch deals, including the new Series 10 models. Note: MacRumors is an affiliate partner with some of these vendors. When you click a link and make a purchase, we may receive a small...

Top Rated Comments

Salty Pirate Avatar
48 months ago
such a bad idea.........


Guys gonna get in trouble gazing at dat ass
Score: 4 Votes (Like | Disagree)
Corsig Avatar
48 months ago

such a bad idea.........


Guys gonna get in trouble gazing at dat ass
Is there another reason to use them?
Score: 4 Votes (Like | Disagree)
CarpalMac Avatar
48 months ago
Imagine being on a train/park bench/public place, being potentially filmed from multiple angles.

Imagine being in a restaurant and you're not sure whether the person you are talking to is focussing on you or the monkey sat on your head.

Two examples of so, so much horribleness. People walking around with cameras sitting on their face, it'd be like a surveillance planet. Just, yuck.
Score: 2 Votes (Like | Disagree)
jjudson Avatar
48 months ago
A whole new bunch of glassholes coming soon...
Score: 2 Votes (Like | Disagree)
CJ Dorschel Avatar
48 months ago
Apple: ”Let’s discontinue the HomePod instead of updating it cause we’re losing money”

also Apple: “Pour more money into the Apple Car and Glasses!”

Over 1000 posts on the HomePod in a day or two and barely anything on the Apple Car and Glasses. Just sayin??‍♂️
Score: 1 Votes (Like | Disagree)
Corsig Avatar
48 months ago

Dem tatas?
I stand corrected
Score: 1 Votes (Like | Disagree)