Members of Apple's PrimeSense Team Patent Method of Interacting With Mac Using Hand Gestures

Apple today was granted a patent originally filed in August 2016, describing a method in which users would be able to control a Mac computer -- and potentially other devices -- using a "non-tactile three dimensional (3D) user interface" (via Patently Apple). The patent's inventor credits go in part to Amir Hoffnung and Jonathan Pokrass, two current Apple employees who joined the company from PrimeSense after Apple acquired it November 2013.

Some of PrimeSense's tech, which was originally used in Microsoft's Kinect devices on Xbox platforms, now resides in the front-facing TrueDepth camera of the iPhone X, and the new patent hints at a potential future where this technology expands in function to Macs as well. Instead of recognizing faces, Apple's patent describes a Mac that recognizes a "gesture by a hand," allowing users to interact with the computer without tactile inputs like a keyboard, mouse, or trackpad.

apple 3d UI patent


The patent includes a variety of gestures that users would use to control the 3D user interface, including what are called "push," (figure 2) "wave," (figure 3) and "up" (figure 5) interactions, which are all grouped into a category of "focus gestures." According to the patent, some of these could be used to perform basic app interactions, like scrolling through a menu, as well as change the state of the system from locked to unlocked.

Gestures described herein include focus gestures and unlock gestures. A focus gesture enables the user to engage (i.e., take control of) an inactive non-tactile 3D user interface. An unlock gesture enables the user to engage a locked non-tactile 3D user interface, as pressing a specific sequence of keys unlocks a locked cellular phone. In some embodiments, the non-tactile 3D user interface conveys visual feedback to the user performing the focus and the unlock gestures.

Examples of unlock gestures include an "up" gesture (e.g., raising hand 30 a specified distance), a sequence of two sequential wave gestures, and a sequence of two sequential push gestures, as described in detail hereinbelow.

Apple has previously been granted patents related to 3D sensing and gesture controls, but the company has yet to release a product that takes advantage of these features. Some of these previous patents included gesture controls on iPads and iPhones as well as on Magic Keyboards, which would allow users to gain access to virtual buttons and potentially streamline certain elements of the user interface.

apple 3d ui patent 2
Of course, with Face ID already launched on iPhone X, it's predicted that Apple will look into adding the facial recognition software into future iMacs and MacBooks before it focuses on a new unlocking system. It's also unclear whether the new 3D user interface patent would include both pieces of technology -- 3D gestures and Face ID -- to further augment a Mac's unlocking process.

As with any patent, the technology in question might not make it to an Apple product in the near future, if at all, but it is an interesting glimpse into what Apple might be planning to do with PrimeSense's technology down the line.

Tag: Patent

Popular Stories

iPhone 17 Pro Blue Feature Tighter Crop

iPhone 17 Pro Launching Later This Year With These 13 New Features

Wednesday April 23, 2025 8:31 am PDT by
While the iPhone 17 Pro and iPhone 17 Pro Max are not expected to launch until September, there are already plenty of rumors about the devices. Below, we recap key changes rumored for the iPhone 17 Pro models as of April 2025: Aluminum frame: iPhone 17 Pro models are rumored to have an aluminum frame, whereas the iPhone 15 Pro and iPhone 16 Pro models have a titanium frame, and the iPhone ...
iphone 17 dummies sonny dickson

iPhone 17 Air Almost as Thin as Its Buttons, New Images Show

Thursday April 24, 2025 2:14 am PDT by
If you missed the video showing dummy models of Apple's all-new super thin iPhone 17 Air that's expected later this year, Sonny Dickson this morning shared some further images of the device in close alignment with the other dummy models in the iPhone 17 lineup, indicating just how thin it is likely to be in comparison. The iPhone 17 Air is expected to be around 5.5mm thick – with a thicker ...
iPhone 17 Air Pastel Feature

iPhone 17 Air Launching Later This Year With These 16 New Features

Thursday April 24, 2025 8:24 am PDT by
While the so-called "iPhone 17 Air" is not expected to launch until September, there are already plenty of rumors about the ultra-thin device. Overall, the iPhone 17 Air sounds like a mixed bag. While the device is expected to have an impressively thin and light design, rumors indicate it will have some compromises compared to iPhone 17 Pro models, including only a single rear camera, a...
Global Close Your Rings Day Pin

Apple Stores Giving Away a Limited-Edition Pin For Free Today

Thursday April 24, 2025 10:15 am PDT by
Starting today, April 24, Apple Stores around the world are giving away a special pin for free to customers who request one, while supplies last. Photo Credit: Filip Chudzinski The enamel pin's design is inspired by the Global Close Your Rings Day award in the Activity app, which Apple Watch users can receive by closing all three Activity rings today. The limited-edition pin is the physical...
Apple Logo Spotlight Blue

White House Hits Back at Apple's Massive EU Fine

Thursday April 24, 2025 5:57 am PDT by
Apple's $570 million fine from the EU has triggered a sharp rebuke from the White House, which called the fine a form of economic extortion, Reuters reports. The fine was announced on Wednesday by the European Commission, following a formal investigation into Apple's compliance with the bloc's Digital Markets Act (DMA), a landmark piece of legislation aimed at curbing the market dominance of ...
AirPods Pro 3 Mock Feature

AirPods Pro 3 Just Months Away – Here's What We Know

Friday April 18, 2025 5:16 am PDT by
Despite being more than two years old, Apple's AirPods Pro 2 still dominate the premium wireless‑earbud space, thanks to a potent mix of top‑tier audio, class‑leading noise cancellation, and Apple's habit of delivering major new features through software updates. With AirPods Pro 3 widely expected to arrive in 2025, prospective buyers now face a familiar dilemma: snap up the proven...
ipad air magic keyboard feature

iPadOS 19 Rumored to Show Mac-Like Menu Bar When Connected to Magic Keyboard

Thursday April 24, 2025 12:09 pm PDT by
When an iPad running iPadOS 19 is connected to a Magic Keyboard, a macOS-like menu bar will appear on the screen, according to the leaker Majin Bu. This change would further blur the lines between the iPad and the Mac. Bloomberg's Mark Gurman previously claimed that iPadOS 19 will be "more like macOS," with unspecified improvements to productivity, multitasking, and app window management,...
apple watch ultra yellow

What's Next for the Apple Watch Ultra 3 and Apple Watch SE 3

Friday April 25, 2025 2:44 pm PDT by
This week marks the 10th anniversary of the Apple Watch, which launched on April 24, 2015. Yesterday, we recapped features rumored for the Apple Watch Series 11, but since 2015, the Apple Watch has also branched out into the Apple Watch Ultra and the Apple Watch SE, so we thought we'd take a look at what's next for those product lines, too. 2025 Apple Watch Ultra 3 Apple didn't update the...

Top Rated Comments

JM Avatar
97 months ago
I want my unlock to be a middle finger. That'll be fun.

Wait, no... I want the middle finger to force-close everything.
Score: 13 Votes (Like | Disagree)
PaddyTB Avatar
97 months ago
How can this ever be novel or non-obvious? US patent system is soooo broken :-(
Score: 7 Votes (Like | Disagree)
jcmeyer5 Avatar
97 months ago
I got a hand gesture for this "technology"....
Score: 7 Votes (Like | Disagree)
zarmanto Avatar
97 months ago
Remind me again, whether or not fictional representations can be used as prior art?

Score: 6 Votes (Like | Disagree)
Lepton Avatar
97 months ago
It should recognize American Sign Language, which would give you a gestural keyboard that is already well worked out, and some people already know it.
Score: 6 Votes (Like | Disagree)
asleep Avatar
97 months ago


Because we want a bunch of lasers in our eyes and tracking our every move?

Think what Google analytics and Amazon can do with that...

We see you're reaching for a pen... how about a uni-ball Vision Rollerball Pens, Fine Point (0.7mm), Black, 12 Count?

Would you like some SweetLeaf Natural Stevia Sweetener to go in you tea?

Oops, looks like you spilled your coffee. How about some Bounty Select-a-Size Paper Towels, the quicker picker upper!

We're noticing a few unruly nose-hairs, we've ordered you a Fancii Professional Nose & Ear Hair Trimmer with LED Light, Water Resistant, Stainless Steel Blades, and Battery Power!
Score: 6 Votes (Like | Disagree)