Apple Researchers Reveal New AI System That Can Beat GPT-4
Apple researchers have developed an artificial intelligence system named ReALM (Reference Resolution as Language Modeling) that aims to radically enhance how voice assistants understand and respond to commands.
In a research paper (via VentureBeat), Apple outlines a new system for how large language models tackle reference resolution, which involves deciphering ambiguous references to on-screen entities, as well as understanding conversational and background context. As a result, ReALM could lead to more intuitive and natural interactions with devices.
Reference resolution is an important part of natural language understanding, enabling users to use pronouns and other indirect references in conversation without confusion. For digital assistants, this capability has historically been a significant challenge, limited by the need to interpret a wide range of verbal cues and visual information. Apple's ReALM system seeks to address this by converting the complex process of reference resolution into a pure language modeling problem. In doing so, it can comprehend references to visual elements displayed on a screen and integrate this understanding into the conversational flow.
ReALM reconstructs the visual layout of a screen using textual representations. This involves parsing on-screen entities and their locations to generate a textual format that captures the screen's content and structure. Apple researchers found that this strategy, combined with specific fine-tuning of language models for reference resolution tasks, significantly outperforms traditional methods, including the capabilities of OpenAI's GPT-4.
ReALM could enable users to interact with digital assistants much more efficiently with reference to what is currently displayed on their screen without the need for precise, detailed instructions. This has the potential to make voice assistants much more useful in a variety of settings, such as helping drivers navigate infotainment systems while driving or assisting users with disabilities by providing an easier and more accurate means of indirect interaction.
Apple has now published several AI research papers. Last month, the company revealed a new method for training large language models that seamlessly integrates both text and visual information. Apple is widely expected to unveil an array of AI features at WWDC in June.
Popular Stories
While the iPhone 17 Pro and iPhone 17 Pro Max are not expected to launch until September, there are already plenty of rumors about the devices.
iPhone 17 Pro concept based on rumors
Below, we recap key changes rumored for the iPhone 17 Pro models as of January 2025:
More aluminum: iPhone 17 Pro models are rumored to have an aluminum frame, whereas the iPhone 15 Pro and iPhone 16 Pro models ...
It was recently reported that new Apple TV and new HomePod mini models will launch this year, and the devices are expected to have one thing in common.
Bloomberg's Mark Gurman last month reported that the new Apple TV and the new HomePod mini will be equipped with Apple's own combined Wi-Fi and Bluetooth chip. Gurman said the chip supports Wi-Fi 6E, so that could end up being a key upgrade...
A new iPhone SE and an iPad 11 might be coming very soon.
In late December, a private account on X with a track record of leaking accurate iOS-related information said devices codenamed "V59" and "J481" will be released alongside iOS 18.3 and iPadOS 18.3. Bloomberg's Mark Gurman has previously reported that "V59" is a new iPhone SE, and that "J481" is a new entry-level iPad.
iOS 15.3, iOS ...
Apple in October 2024 overhauled its 14-inch and 16-inch MacBook Pro models, adding M4, M4 Pro, and M4 Max chips, Thunderbolt 5 ports on higher-end models, display changes, and more. That's quite a lot of updates in one go, but if you think this means a further major refresh for the MacBook Pro is now several years away, think again.
Bloomberg's Mark Gurman has said he expects only a small...
If you've been hearing a chiming sound from your AirPods Pro 2 case when the AirPods are charging, it's a feature that Apple added with the launch of Hearing Health last year.
In a support guide, Apple says that the AirPods Pro may play a sound every so often while in the case to ensure the microphones and speakers are working as intended. From Apple:
To help ensure that your AirPods...
Apple today released iOS 18.2.1 and iPadOS 18.2.1, minor updates to the iOS 18 and iPadOS 18 operating systems. iOS 18.2.1 and iPadOS 18.2.1 come almost a month after Apple released iOS 18.2 and iPadOS 18.2.
The new software can be downloaded on eligible iPhones and iPads over-the-air by going to Settings > General > Software Update.
According to Apple's release notes, iOS 18.2.1...
Apple's annual "Back to School" or "Back to Uni" promotion has returned this week in Australia, New Zealand, Brazil, and South Korea.
From January 8 through March 13, qualifying higher-education students and staff in these countries can receive free AirPods 4 with Active Noise Cancellation with the purchase of any new MacBook Air, MacBook Pro, or iMac, or a free Apple Pencil Pro or Apple...
LG has shown off a new Ultrafine 6K monitor at CES 2025. The 32-inch display is the first of its kind to support Thunderbolt 5, which Apple introduced late last year with the launch of new Mac mini and MacBook Pro models powered by M4 Pro chips.
Details are scant, but we do know that the LG UltraFine 6K monitor (model 32U990A) features a Nano IPS Black panel, delivering a wide color gamut...
