Apple Researchers Reveal New AI System That Can Beat GPT-4
Apple researchers have developed an artificial intelligence system named ReALM (Reference Resolution as Language Modeling) that aims to radically enhance how voice assistants understand and respond to commands.
In a research paper (via VentureBeat), Apple outlines a new system for how large language models tackle reference resolution, which involves deciphering ambiguous references to on-screen entities, as well as understanding conversational and background context. As a result, ReALM could lead to more intuitive and natural interactions with devices.
Reference resolution is an important part of natural language understanding, enabling users to use pronouns and other indirect references in conversation without confusion. For digital assistants, this capability has historically been a significant challenge, limited by the need to interpret a wide range of verbal cues and visual information. Apple's ReALM system seeks to address this by converting the complex process of reference resolution into a pure language modeling problem. In doing so, it can comprehend references to visual elements displayed on a screen and integrate this understanding into the conversational flow.
ReALM reconstructs the visual layout of a screen using textual representations. This involves parsing on-screen entities and their locations to generate a textual format that captures the screen's content and structure. Apple researchers found that this strategy, combined with specific fine-tuning of language models for reference resolution tasks, significantly outperforms traditional methods, including the capabilities of OpenAI's GPT-4.
ReALM could enable users to interact with digital assistants much more efficiently with reference to what is currently displayed on their screen without the need for precise, detailed instructions. This has the potential to make voice assistants much more useful in a variety of settings, such as helping drivers navigate infotainment systems while driving or assisting users with disabilities by providing an easier and more accurate means of indirect interaction.
Apple has now published several AI research papers. Last month, the company revealed a new method for training large language models that seamlessly integrates both text and visual information. Apple is widely expected to unveil an array of AI features at WWDC in June.
Popular Stories
Apple is about to release iOS 26.2, the second major point update for iPhones since iOS 26 was rolled out in September, and there are at least 15 notable changes and improvements worth checking out. We've rounded them up below.
Apple is expected to roll out iOS 26.2 to compatible devices sometime between December 8 and December 16. When the update drops, you can check Apple's servers for the ...
Apple has ordered 22 million OLED panels from Samsung Display for the first foldable iPhone, signaling a significantly larger production target than the display industry had previously anticipated, ET News reports.
In the now-seemingly deleted report, ET News claimed that Samsung plans to mass-produce 11 million inward-folding OLED displays for Apple next year, as well as 11 million...
Apple is actively testing under-screen Face ID for next year's iPhone 18 Pro models using a special "spliced micro-transparent glass" window built into the display, claims a Chinese leaker.
According to "Smart Pikachu," a Weibo account that has previously shared accurate supply-chain details on Chinese Android hardware, Apple is testing the special glass as a way to let the TrueDepth...
Apple today seeded the second release candidate version of iOS 26.2 to developers and public beta testers, with the software coming one week after Apple seeded the first RC. The release candidate represents the final version iOS 26.2 that will be provided to the public if no further bugs are found.
Registered developers and public beta testers can download the betas from the Settings app on...
Apple's iPhone development roadmap runs several years into the future and the company is continually working with suppliers on several successive iPhone models at the same time, which is why we often get rumored features months ahead of launch. The iPhone 18 series is no different, and we already have a good idea of what to expect for the iPhone 18 Pro and iPhone 18 Pro Max.
One thing worth...
Google Maps on iOS quietly gained a new feature recently that automatically recognizes where you've parked your vehicle and saves the location for you.
Announced on LinkedIn by Rio Akasaka, Google Maps' senior product manager, the new feature auto-detects your parked location even if you don't use the parking pin function, saves it for up to 48 hours, and then automatically removes it once...
Apple's chipmaking chief Johny Srouji has reportedly indicated that he plans to continue working for the company for the foreseeable future.
"I love my team, and I love my job at Apple, and I don't plan on leaving anytime soon," said Srouji, in a memo obtained by Bloomberg's Mark Gurman.
Here is Srouji's full memo, as shared by Bloomberg:I know you've been reading all kind of rumors and...
Apple's senior vice president of hardware technologies Johny Srouji could be the next leading executive to leave the company amid an alarming exodus of leading employees, Bloomberg's Mark Gurman reports.
Srouji apparently recently told CEO Tim Cook that he is "seriously considering leaving" in the near future. He intends to join another company if he departs. Srouji leads Apple's chip design ...
Apple and Google are teaming up to make it easier for users to switch between iPhone and Android smartphones, according to 9to5Google. There is a new Android Canary build available today that simplifies data transfer between two smartphones, and Apple is going to implement the functionality in an upcoming iOS 26 beta.
Apple already has a Move to iOS app for transferring data from an Android...
Intel is expected to begin supplying some Mac and iPad chips in a few years, and the latest rumor claims the partnership might extend to the iPhone.
In a research note with investment firm GF Securities this week, obtained by MacRumors, analyst Jeff Pu said he and his colleagues "now expect" Intel to reach a supply deal with Apple for at least some non-pro iPhone chips starting in 2028....
