Apple Researchers Reveal New AI System That Can Beat GPT-4
Apple researchers have developed an artificial intelligence system named ReALM (Reference Resolution as Language Modeling) that aims to radically enhance how voice assistants understand and respond to commands.
In a research paper (via VentureBeat), Apple outlines a new system for how large language models tackle reference resolution, which involves deciphering ambiguous references to on-screen entities, as well as understanding conversational and background context. As a result, ReALM could lead to more intuitive and natural interactions with devices.
Reference resolution is an important part of natural language understanding, enabling users to use pronouns and other indirect references in conversation without confusion. For digital assistants, this capability has historically been a significant challenge, limited by the need to interpret a wide range of verbal cues and visual information. Apple's ReALM system seeks to address this by converting the complex process of reference resolution into a pure language modeling problem. In doing so, it can comprehend references to visual elements displayed on a screen and integrate this understanding into the conversational flow.
ReALM reconstructs the visual layout of a screen using textual representations. This involves parsing on-screen entities and their locations to generate a textual format that captures the screen's content and structure. Apple researchers found that this strategy, combined with specific fine-tuning of language models for reference resolution tasks, significantly outperforms traditional methods, including the capabilities of OpenAI's GPT-4.
ReALM could enable users to interact with digital assistants much more efficiently with reference to what is currently displayed on their screen without the need for precise, detailed instructions. This has the potential to make voice assistants much more useful in a variety of settings, such as helping drivers navigate infotainment systems while driving or assisting users with disabilities by providing an easier and more accurate means of indirect interaction.
Apple has now published several AI research papers. Last month, the company revealed a new method for training large language models that seamlessly integrates both text and visual information. Apple is widely expected to unveil an array of AI features at WWDC in June.
Popular Stories
Apple today seeded the second release candidate versions of upcoming iOS 18.2, iPadOS 18.2, and macOS 15.2 updates to developers and public beta testers for testing purposes, a week after releasing the first RCs. The first iOS 18.2 RC had a build number of 22C150, while the second RC's build number is 22C151. Release candidates represent the final version of beta software that's expected to see a ...
Apple today announced that iOS 18.2, iPadOS 18.2, and macOS Sequoia 15.2 will be released today following more than six weeks of beta testing.
For the iPhone 15 Pro and iPhone 16 models, the update introduces additional Apple Intelligence features, including Genmoji for creating custom emoji, Image Playground and Image Wand for generating images, and ChatGPT integration for Siri. There is...
Apple's forthcoming iPhone SE 4 will feature a single 48-megapixel rear camera and a 12-megapixel TrueDepth camera on the front, according to details revealed in a new Korean supply chain report.
ET News reports that Korea-based LG Innotek is the main supplier of the front and rear camera modules for the more budget-friendly ~$400 device, which is expected to launch in the first quarter of...
Apple seeded the release candidate version of iOS 18.2 today, which means it's going to see a public launch imminently. Release candidates represent the final version of new software that will be provided to the public should no last minute bugs be found, and Apple includes release notes with the RC launch.
The iOS 18.2 release notes provide a look at all of the new features that are coming...
Apple plans to remove the notch from the MacBook Pro in a few years from now, according to a roadmap shared by research firm Omdia.
The roadmap shows that 14-inch and 16-inch MacBook Pro models released in 2026 will have a hole-punch camera at the top of the display, instead of a notch. It is unclear if there would simply be a pinhole in the display, or if Apple would expand the iPhone's...
Apple is set to release iOS 18.2 in the second week of December, bringing the second round of Apple Intelligence features to iPhone 15 Pro and iPhone 16 models. This update brings several major advancements to Apple's AI integration, including completely new image generation tools and a range of Visual Intelligence-based enhancements. There are a handful of new non-AI related feature controls...
Despite being released over two years ago, Apple's AirPods Pro 2 continue to dominate the wireless earbud market. However, with the AirPods Pro 3 expected to launch sometime in 2025, anyone thinking of buying Apple's premium earbuds may be wondering if the next generation is worth holding out for.
Apart from their audio and noise-canceling performance, which are generally regarded as...
Norwegian payment service Vipps has become the world's first company to launch a competing tap-to-pay solution to Apple Pay on iPhone, following Apple's agreement with European regulators to open up its NFC technology to third parties.
Starting December 9, Vipps users in Norway can make contactless payments in stores using their iPhones. The service initially supports customers of SpareBank...
