Starting in iOS 14 and macOS Big Sur, developers will be able to add the capability to detect human body and hand poses in photos and videos to their apps using Apple's updated Vision framework, as explained in this WWDC 2020 session.
This functionality will allow apps to analyze the poses, movements, and gestures of people, enabling a wide variety of potential features. Apple provides some examples, including a fitness app that could automatically track the exercise a user performs, a safety-training app that could help employees use correct ergonomics, and a media-editing app that could find photos or videos based on pose similarity.
Hand pose detection in particular promises to deliver a new form of interaction with apps. Apple's demonstration showed a person holding their thumb and index finger together and then being able to draw in an iPhone app without touching the display.
Additionally, apps could use the framework to overlay emoji or graphics on a user's hands that mirror the specific gesture, such as a peace sign.
Another example is a camera app that automatically triggers photo capture when it detects the user making a specific hand gesture in the air.
The framework is capable of detecting multiple hands or bodies in one scene, but the algorithms might not work as well with people who are wearing gloves, bent over, facing upside down, or wearing overflowing or robe-like clothing. The algorithm can also experience difficulties if a person is close to edge of the screen or partially obstructed.
Similar functionality is already available through ARKit, but it is limited to augmented reality sessions and only works with the rear-facing camera on compatible iPhone and iPad models. With the updated Vision framework, developers have many more possibilities.
iOS 19 will not be available on the iPhone XR, iPhone XS, or the iPhone XS Max, according a private account on social media site X that has accurately provided information on device compatibility in the past.
The iPhone XR, iPhone XS, and iPhone XS Max all have an A12 Bionic chip, so it looks like iOS 19 will discontinue support for that chip. All other iPhones that run iOS 18 are expected...
Apple today released watchOS 11.4, the fourth major update to the operating system that runs on the Apple Watch. watchOS 11.4 is compatible with the Apple Watch Series 6 and later, all Apple Watch Ultra models, and the Apple Watch SE 2.
watchOS 11.4 can be downloaded on a connected iPhone by opening up the Apple Watch app and going to General > Software Update. To install the new software,...
Apple today released new firmware updates for all AirPods 4 and AirPods Pro 2 models. The new firmware is version 7E93, up from the 7B21 firmware that was installed on the AirPods Pro 2 and the 7B20 firmware available on the AirPods 4 and AirPods 4 with ANC.
It is not immediately clear what new features or changes are included in the new firmware, but we'll update this article should we find ...
Apple today released iOS 18.4 and iPadOS 18.4, the fourth major updates to the iOS 18 and iPadOS 18 operating system updates that came out last year. iOS 18.4 and iPadOS 18.4 come two months after Apple released iOS 18.3 and iPadOS 18.3.
Subscribe to the MacRumors YouTube channel for more videos.
The new software can be downloaded on eligible iPhones and iPads over-the-air by going to...
While the iPhone 17 Pro and iPhone 17 Pro Max are not expected to launch until September, there are already plenty of rumors about the devices.
Below, we recap key changes rumored for the iPhone 17 Pro models as of March 2025:
Aluminum frame: iPhone 17 Pro models are rumored to have an aluminum frame, whereas the iPhone 15 Pro and iPhone 16 Pro models have a titanium frame, and the iPhone ...
Visa wants to pay Apple approximately $100 million to be the new payment network for the Apple Card, reports The Wall Street Journal. As of right now, the Apple Card is on the Mastercard payment network, but that is set to change because Apple is ending its partnership with Goldman Sachs.
Both American Express and Visa are vying to replace Mastercard as Apple's card services provider, while...
Wednesday April 2, 2025 10:11 am PDT by Juli Clover
Apple today seeded the first betas of upcoming iOS 18.5 and iPadOS 18.5 updates to developers for testing purposes, with the software coming just two days after Apple released iOS 18.4 and iPadOS 18.4.
iOS 18.5 and iPadOS 18.5 can be downloaded from the Settings app on a compatible device by going to General > Software Update.
We don't yet know what Apple is introducing in the iOS 18.5...
Honestly, this seems like the kinda stuff that'd make Apple AR compelling—being able to draw in midair means you’d also be able to navigate an interface in midair with just your hands.
Using AR/VR without bringing a controller everywhere seems analogous to what set the iPhone apart from other touchscreen phones in 2007; you didn’t need a stylus.
The framework is capable of detecting multiple hands or bodies in one scene, but the algorithms might not work as well with people who are wearing gloves, bent over, facing upside down, or wearing overflowing or robe-like clothing.
I don't think the wizarding community is going to be too happy about this...
