Apple Researchers Reveal New AI System That Can Beat GPT-4
Apple researchers have developed an artificial intelligence system named ReALM (Reference Resolution as Language Modeling) that aims to radically enhance how voice assistants understand and respond to commands.
In a research paper (via VentureBeat), Apple outlines a new system for how large language models tackle reference resolution, which involves deciphering ambiguous references to on-screen entities, as well as understanding conversational and background context. As a result, ReALM could lead to more intuitive and natural interactions with devices.
Reference resolution is an important part of natural language understanding, enabling users to use pronouns and other indirect references in conversation without confusion. For digital assistants, this capability has historically been a significant challenge, limited by the need to interpret a wide range of verbal cues and visual information. Apple's ReALM system seeks to address this by converting the complex process of reference resolution into a pure language modeling problem. In doing so, it can comprehend references to visual elements displayed on a screen and integrate this understanding into the conversational flow.
ReALM reconstructs the visual layout of a screen using textual representations. This involves parsing on-screen entities and their locations to generate a textual format that captures the screen's content and structure. Apple researchers found that this strategy, combined with specific fine-tuning of language models for reference resolution tasks, significantly outperforms traditional methods, including the capabilities of OpenAI's GPT-4.
ReALM could enable users to interact with digital assistants much more efficiently with reference to what is currently displayed on their screen without the need for precise, detailed instructions. This has the potential to make voice assistants much more useful in a variety of settings, such as helping drivers navigate infotainment systems while driving or assisting users with disabilities by providing an easier and more accurate means of indirect interaction.
Apple has now published several AI research papers. Last month, the company revealed a new method for training large language models that seamlessly integrates both text and visual information. Apple is widely expected to unveil an array of AI features at WWDC in June.
Popular Stories
While the iPhone 17 Pro and iPhone 17 Pro Max are not expected to launch until September, there are already plenty of rumors about the devices.
Below, we recap key changes rumored for the iPhone 17 Pro models as of April 2025:
Aluminum frame: iPhone 17 Pro models are rumored to have an aluminum frame, whereas the iPhone 15 Pro and iPhone 16 Pro models have a titanium frame, and the iPhone ...
Apple may have updated several iPads and Macs late last year and early this year, but there are still multiple new devices that we're looking forward to seeing in 2025. Most will come in September or October, but there could be a few surprises before then.
We've rounded up a list of everything that we're still waiting to see from Apple in 2025.
iPhone 17, 17 Air, and 17 Pro - We get...
Apple's upcoming foldable iPhone (or "iPhone Fold") will feature two screens as part of its book-style design, and a Chinese leaker claims to know the resolutions for both of them.
According to the Weibo-based account Digital Chat Station, the inner display, which is approximately 7.76 inches, will use a 2,713 x 1,920 resolution and feature "under-screen camera technology." Meanwhile, the...
A common complaint about the iPad Pro is that the iPadOS software platform fails to fully take advantage of the device's powerful hardware.
That could soon change.
Bloomberg's Mark Gurman today said that iPadOS 19 will be "more like macOS."
Gurman said that iPadOS 19 will be "more like a Mac" in three ways:Improved productivity
Improved multitasking
Improved app window management...
Apple in October 2024 overhauled its 14-inch and 16-inch MacBook Pro models, adding M4, M4 Pro, and M4 Max chips, Thunderbolt 5 ports on higher-end models, display changes, and more. That's quite a lot of updates in one go, but if you think this means a further major refresh for the MacBook Pro is now several years away, think again.
Bloomberg's Mark Gurman has said he expects only a small...
Apple is working on a new version of the Vision Pro with two key advantages over the current model, according to Bloomberg's Mark Gurman.
Specifically, in his Power On newsletter today, Gurman said Apple is developing a new headset that is both lighter and less expensive than the current Vision Pro, which starts at $3,499 in the U.S. and weighs up to 1.5 pounds.
Gurman said Apple is also...
On this week's episode of The MacRumors Show, we catch up on the latest iOS 19 and watchOS 12 rumors, upcoming devices, and more.
Subscribe to The MacRumors Show YouTube channel for more videos
Detailed new renders from leaker Jon Prosser claim to provide the best look yet at the complete redesign rumored to arrive in iOS 19, showing more rounded elements, lighting effects, translucency, and...
It was a big week for leaks and rumors in the Apple world, with fresh claims about iOS 19, the iPhone 17 Pro, and even the 20th anniversary iPhone coming a couple of years from now.
Sources also spilled the tea on the inner turmoil at Apple around the Apple Intelligence-driven Siri revamp that has seen significant delays, so read on below for all the details on these stories and more!
iOS ...
Apple and other electronics manufacturers have received a break from Trump's reciprocal tariffs, with the U.S. Customs and Border Protection agency sharing a long list of products excluded from the levies last night.
iPhones, Macs, iPads, Apple Watch, and other Apple devices will not be subject to the 125 percent tariffs that have been put in place on imported Chinese goods, nor will Apple...
