Members of Apple's PrimeSense Team Patent Method of Interacting With Mac Using Hand Gestures

Apple today was granted a patent originally filed in August 2016, describing a method in which users would be able to control a Mac computer -- and potentially other devices -- using a "non-tactile three dimensional (3D) user interface" (via Patently Apple). The patent's inventor credits go in part to Amir Hoffnung and Jonathan Pokrass, two current Apple employees who joined the company from PrimeSense after Apple acquired it November 2013.

Some of PrimeSense's tech, which was originally used in Microsoft's Kinect devices on Xbox platforms, now resides in the front-facing TrueDepth camera of the iPhone X, and the new patent hints at a potential future where this technology expands in function to Macs as well. Instead of recognizing faces, Apple's patent describes a Mac that recognizes a "gesture by a hand," allowing users to interact with the computer without tactile inputs like a keyboard, mouse, or trackpad.

apple 3d UI patent


The patent includes a variety of gestures that users would use to control the 3D user interface, including what are called "push," (figure 2) "wave," (figure 3) and "up" (figure 5) interactions, which are all grouped into a category of "focus gestures." According to the patent, some of these could be used to perform basic app interactions, like scrolling through a menu, as well as change the state of the system from locked to unlocked.

Gestures described herein include focus gestures and unlock gestures. A focus gesture enables the user to engage (i.e., take control of) an inactive non-tactile 3D user interface. An unlock gesture enables the user to engage a locked non-tactile 3D user interface, as pressing a specific sequence of keys unlocks a locked cellular phone. In some embodiments, the non-tactile 3D user interface conveys visual feedback to the user performing the focus and the unlock gestures.

Examples of unlock gestures include an "up" gesture (e.g., raising hand 30 a specified distance), a sequence of two sequential wave gestures, and a sequence of two sequential push gestures, as described in detail hereinbelow.

Apple has previously been granted patents related to 3D sensing and gesture controls, but the company has yet to release a product that takes advantage of these features. Some of these previous patents included gesture controls on iPads and iPhones as well as on Magic Keyboards, which would allow users to gain access to virtual buttons and potentially streamline certain elements of the user interface.

apple 3d ui patent 2
Of course, with Face ID already launched on iPhone X, it's predicted that Apple will look into adding the facial recognition software into future iMacs and MacBooks before it focuses on a new unlocking system. It's also unclear whether the new 3D user interface patent would include both pieces of technology -- 3D gestures and Face ID -- to further augment a Mac's unlocking process.

As with any patent, the technology in question might not make it to an Apple product in the near future, if at all, but it is an interesting glimpse into what Apple might be planning to do with PrimeSense's technology down the line.

Tag: Patent

Popular Stories

2024 iPhone Boxes Feature

iPhones Could Cost Up to $2,300 in the U.S. Due to Tariffs, Analyst Says

Friday April 4, 2025 9:30 am PDT by
U.S. President Donald Trump on Wednesday announced that steep tariffs will be applied to imports from many countries, starting April 9. The tariffs could have a significant impact on Apple, as the company assembles the majority of iPhones in China, and products imported to the U.S. from China will be subject to a 54% tariff. iPhone prices could increase by up to 43% in the U.S. due to the...
iphone 16 pro colors 1

Is Now the Time to Upgrade Apple Devices Before Tariffs Lead to Price Increases?

Friday April 4, 2025 3:41 pm PDT by
If you have an older Apple device that you've been considering upgrading, you're probably wondering how the newly announced tariffs might impact prices going forward, and whether it's worth buying now before there's a price hike. Given analyst and economist responses to the tariffs, market panic, and Trump's stance on the current financial chaos, the answer is that making a purchase...
iOS 19 Mock WWDC25 Feature

iOS 19 Expected to Run on These iPhones

Monday March 31, 2025 5:28 pm PDT by
iOS 19 will not be available on the iPhone XR, iPhone XS, or the iPhone XS Max, according a private account on social media site X that has accurately provided information on device compatibility in the past. The iPhone XR, iPhone XS, and iPhone XS Max all have an A12 Bionic chip, so it looks like iOS 19 will discontinue support for that chip. All other iPhones that run iOS 18 are expected...
iPhone 17 Pro 34ths Perspective

iPhone 17 Pro: New 48MP Telephoto Lens May Change How Zoom Works

Thursday April 3, 2025 5:11 am PDT by
Apple is reportedly planning a major upgrade to the Telephoto camera in the iPhone 17 Pro, and while it may seem like a step back on paper, the change could actually improve real-world usability, if one leaker's claims are anything to go by. According to Majin Bu, the iPhone 17 Pro will feature a new Telephoto lens with a 48MP sensor, up from the current 12MP sensor found in the iPhone 16...
iOS 18

12 New Things Your iPhone Can Do in iOS 18.4

Tuesday April 1, 2025 4:06 am PDT by
Apple has released iOS 18.4, bringing further refinements to Apple Intelligence features, a neat new capability to iPhone 15 Pro devices, new emoji, and more. While not quite as packed with new features as Apple's preceding iOS 18 point releases, iOS 18.4 still introduces enhancements that aim to make your iPhone smarter and more intuitive. Below, we've listed 12 new things your ‌iPhone‌ ...
iPhone 17 Pro 34ths Perspective

iPhone 17 Pro Launching Later This Year With These 10 New Features

Sunday March 23, 2025 10:00 am PDT by
While the iPhone 17 Pro and iPhone 17 Pro Max are not expected to launch until September, there are already plenty of rumors about the devices. Below, we recap key changes rumored for the iPhone 17 Pro models as of March 2025: Aluminum frame: iPhone 17 Pro models are rumored to have an aluminum frame, whereas the iPhone 15 Pro and iPhone 16 Pro models have a titanium frame, and the iPhone ...

Top Rated Comments

JM Avatar
96 months ago
I want my unlock to be a middle finger. That'll be fun.

Wait, no... I want the middle finger to force-close everything.
Score: 13 Votes (Like | Disagree)
PaddyTB Avatar
96 months ago
How can this ever be novel or non-obvious? US patent system is soooo broken :-(
Score: 7 Votes (Like | Disagree)
jcmeyer5 Avatar
96 months ago
I got a hand gesture for this "technology"....
Score: 7 Votes (Like | Disagree)
zarmanto Avatar
96 months ago
Remind me again, whether or not fictional representations can be used as prior art?

Score: 6 Votes (Like | Disagree)
Lepton Avatar
96 months ago
It should recognize American Sign Language, which would give you a gestural keyboard that is already well worked out, and some people already know it.
Score: 6 Votes (Like | Disagree)
asleep Avatar
96 months ago


Because we want a bunch of lasers in our eyes and tracking our every move?

Think what Google analytics and Amazon can do with that...

We see you're reaching for a pen... how about a uni-ball Vision Rollerball Pens, Fine Point (0.7mm), Black, 12 Count?

Would you like some SweetLeaf Natural Stevia Sweetener to go in you tea?

Oops, looks like you spilled your coffee. How about some Bounty Select-a-Size Paper Towels, the quicker picker upper!

We're noticing a few unruly nose-hairs, we've ordered you a Fancii Professional Nose & Ear Hair Trimmer with LED Light, Water Resistant, Stainless Steel Blades, and Battery Power!
Score: 6 Votes (Like | Disagree)