Apple Researchers Reveal New AI System That Can Beat GPT-4
Apple researchers have developed an artificial intelligence system named ReALM (Reference Resolution as Language Modeling) that aims to radically enhance how voice assistants understand and respond to commands.
In a research paper (via VentureBeat), Apple outlines a new system for how large language models tackle reference resolution, which involves deciphering ambiguous references to on-screen entities, as well as understanding conversational and background context. As a result, ReALM could lead to more intuitive and natural interactions with devices.
Reference resolution is an important part of natural language understanding, enabling users to use pronouns and other indirect references in conversation without confusion. For digital assistants, this capability has historically been a significant challenge, limited by the need to interpret a wide range of verbal cues and visual information. Apple's ReALM system seeks to address this by converting the complex process of reference resolution into a pure language modeling problem. In doing so, it can comprehend references to visual elements displayed on a screen and integrate this understanding into the conversational flow.
ReALM reconstructs the visual layout of a screen using textual representations. This involves parsing on-screen entities and their locations to generate a textual format that captures the screen's content and structure. Apple researchers found that this strategy, combined with specific fine-tuning of language models for reference resolution tasks, significantly outperforms traditional methods, including the capabilities of OpenAI's GPT-4.
ReALM could enable users to interact with digital assistants much more efficiently with reference to what is currently displayed on their screen without the need for precise, detailed instructions. This has the potential to make voice assistants much more useful in a variety of settings, such as helping drivers navigate infotainment systems while driving or assisting users with disabilities by providing an easier and more accurate means of indirect interaction.
Apple has now published several AI research papers. Last month, the company revealed a new method for training large language models that seamlessly integrates both text and visual information. Apple is widely expected to unveil an array of AI features at WWDC in June.
Popular Stories
Apple is about to release iOS 26.2, the second major point update for iPhones since iOS 26 was rolled out in September, and there are at least 15 notable changes and improvements worth checking out. We've rounded them up below.
Apple is expected to roll out iOS 26.2 to compatible devices sometime between December 8 and December 16. When the update drops, you can check Apple's servers for the ...
Intel is expected to begin supplying some Mac and iPad chips in a few years, and the latest rumor claims the partnership might extend to the iPhone.
In a research note with investment firm GF Securities this week, obtained by MacRumors, analyst Jeff Pu said he and his colleagues "now expect" Intel to reach a supply deal with Apple for at least some non-pro iPhone chips starting in 2028....
A U.S. appeals court has upheld a temporary restraining order that prevents OpenAI and Jony Ive's new hardware venture from using the name "io" for products similar to those planned by AI audio startup iyO, Bloomberg Law reports.
iyO sued OpenAI earlier this year after the latter announced its partnership with Ive's new firm, arguing that OpenAI's planned "io" branding was too close to its...
Apple's iPhone development roadmap runs several years into the future and the company is continually working with suppliers on several successive iPhone models at the same time, which is why we often get rumored features months ahead of launch. The iPhone 18 series is no different, and we already have a good idea of what to expect for the iPhone 18 Pro and iPhone 18 Pro Max.
One thing worth...
In a statement shared with Bloomberg on Wednesday, Apple confirmed that its software design chief Alan Dye will be leaving. Apple said Dye will be succeeded by Stephen Lemay, who has been a software designer at the company since 1999.
Meta CEO Mark Zuckerberg announced that Dye will lead a new creative studio within the company's AR/VR division Reality Labs.
On his blog Daring Fireball,...
Apple is expected to launch a new foldable iPhone next year, based on multiple rumors and credible sources. The long-awaited device has been rumored for years now, but signs increasingly suggest that 2026 could indeed be the year that Apple releases its first foldable device.
Subscribe to the MacRumors YouTube channel for more videos.
Below, we've collated an updated set of key details that ...
There is uncertainty about Apple's head of hardware engineering John Ternus succeeding Tim Cook as CEO, The Information reports. Some former Apple executives apparently hope that a new "dark-horse" candidate will emerge.
Ternus is considered to be the most likely candidate to succeed Cook as CEO. The report notes that he is more likely to become CEO than software head chief Craig Federighi, ...
Apple is encouraging iPhone users who are still running iOS 18 to upgrade to iOS 26 by making the iOS 26 software upgrade option more prominent.
Since iOS 26 launched in September, it has been displayed as an optional upgrade at the bottom of the Software Update interface in the Settings app. iOS 18 has been the default operating system option, and users running iOS 18 have seen iOS 18...
Apple today seeded the release candidate versions of upcoming iOS 26.2 and iPadOS 26.2 updates to developers and public beta testers, with the software coming two weeks after Apple seeded the third betas. The release candidates represent the final versions of iOS 26.2 and iPadOS 26.2 that will be provided to the public if no further bugs are found during this final week of testing....
Apple's senior vice president of hardware technologies Johny Srouji could be the next leading executive to leave the company amid an alarming exodus of leading employees, Bloomberg's Mark Gurman reports.
Srouji apparently recently told CEO Tim Cook that he is "seriously considering leaving" in the near future. He intends to join another company if he departs. Srouji leads Apple's chip design ...
