AI Companies Reportedly Struggling to Improve Latest Models

Leading artificial intelligence companies including OpenAI, Google, and Anthropic are facing "diminishing returns" from their costly efforts to build newer AI models, according to a new Bloomberg report. The stumbling blocks appear to be growing in size as Apple continues a phased rollout of its own AI features through Apple Intelligence.

chatgpt for iphone feature
OpenAI's latest model, known internally as Orion, has reportedly fallen short of the company's performance expectations, particularly in handling coding tasks. The model is said to be lacking the significant improvements over existing systems when compared to the gains GPT-4 made versus its predecessor.

Google is also reportedly facing similar obstacles with its upcoming Gemini software, while Anthropic has delayed the release of its anticipated Claude 3.5 Opus model. Industry experts who spoke to Bloomberg attributed the challenges to the increasing difficulty in finding "new, untapped sources of high-quality, human-made training data" and the enormous costs associated with developing and operating new models concurrently with existing ones.

Silicon Valley's belief that more computing power, data, and larger models will inevitably lead to better performance, and ultimately the holy grail – artificial general intelligence (AGI) – could be based on false assumptions, suggests the report. Consequently, companies are now exploring alternative approaches, including further post-training (incorporating human feedback to improve responses and refining the tone) and developing AI tools called agents that can perform targeted tasks, such as booking flights or sending emails on a user's behalf.

"The AGI bubble is bursting a little bit," said Margaret Mitchell, chief ethics scientist at AI startup Hugging Face. She told Bloomberg that "different training approaches" may be needed to make AI models work really well on a variety of tasks. Other experts who spoke to the outlet echoed Mitchell's sentiment.

How much impact these challenges will have on Apple's approach is unclear, though Apple Intelligence is more focused in comparison, and the company uses internal large language models (LLMs) grounded in privacy. Apple's AI services mainly operate on-device, while the company's Private Cloud Compute encrypted servers are only pinged for tasks requiring more advanced processing power.

Apple is integrating AI capabilities into existing products and services, including writing tools, Siri improvements, and image generation features, so it can't be said to be competing directly in the LLM space. However, Apple has agreed a partnership with OpenAI that allows Siri to optionally hand off more open-ended queries to ChatGPT. Apple has also reportedly held discussions with other LLM companies about similar outsourcing partnerships.

It's possible that the challenges faced by major AI companies pursuing breakthrough general-purpose AI models could ultimately validate Apple's more conservative strategy of developing specific AI features that enhance the user experience. In that sense, its privacy-first policy may not be the straitjacket it first seemed. Apple plans to expand Apple Intelligence features next month with the release of iOS 18.2 and then via further updates through 2025.

Popular Stories

AirPods Pro Firmware Feature

Apple Releases Firmware Updates for AirPods Pro 2 and AirPods 4

Monday November 11, 2024 11:28 am PST by
Apple today released firmware updates for both AirPods 4 models (version number 7B20) and the AirPods Pro 2 with both Lightning and USB-C charging cases (version number 7B21). All of these AirPods models were previously on firmware version 7B19. It is not immediately clear what new features or changes are included in firmware versions 7B20 and 7B21, but we will update this story if we find...
New Things Your iPhone Can Do in iOS 18

18 New Things Your iPhone Can Do in iOS 18.2

Wednesday November 13, 2024 2:09 am PST by
Apple is set to release iOS 18.2 next month, bringing the second round of Apple Intelligence features to iPhone 15 Pro and iPhone 16 models. This update brings several major advancements to Apple's AI integration, including completely new image generation tools and a range of Visual Intelligence-based enhancements. There are a handful of new non-AI related feature controls incoming as well....
iPhone SE 4 Thumb 1

iPhone SE 4 Camera Modules to Enter Mass Production Next Month

Tuesday November 12, 2024 2:56 am PST by
Apple's camera module supplier for the upcoming iPhone SE 4 is set to begin mass production of the components in December, according to a new report coming out of Korea. Economic newspaper Ajunews reports that LG Innotek will supply the front camera module for the budget-friendly fourth-generation device. Final tests are now said to be underway, with mass production of the module following...
iphone 6 thickness

iPhone 17 'Air' May Not Be Much Thinner Than iPhone 6

Monday November 11, 2024 5:18 am PST by
Next year's iPhone 17 "Air" model may not be as thin as Apple planned, according to a rumor originating in Korea. According to the news aggregator account "yeux1122" on Naver, citing industry sources, Apple has run into problems making the new iPhone 17 model sufficiently thin. The device's reduced thickness is apparently dependent on manufacturing a battery with a thinner substrate, but...
new mac holiday

The Best Early Black Friday Mac Deals

Monday November 11, 2024 7:45 am PST by
Black Friday is getting closer, and prices on MacBook Pro, MacBook Air, iMac, and Mac mini computers have started to drop as the shopping holiday nears. These deals include the latest models of the M4 MacBook Pro and iMac. Note: MacRumors is an affiliate partner with some of these vendors. When you click a link and make a purchase, we may receive a small payment, which helps us keep the site...
iOS 18

Here's Everything New in iOS 18.2 Beta 3

Monday November 11, 2024 4:53 pm PST by
Apple seeded the third betas of iOS 18.2 and iPadOS 18.2 to developers for testing today. While the third betas of each update are minor relative to the first two betas, there are still a handful of changes across the Photos app, TV app, and more. A corresponding iOS 18.2 public beta with these changes will likely be released later this week, and Apple previously confirmed that the software...
General Final Cut Pro Feature

Apple Likely to Announce Final Cut Pro Update This Week With These New Features

Sunday November 10, 2024 12:13 pm PST by
In its announcement video for the new Mac mini last month, Apple teased an "upcoming" version of Final Cut Pro for the Mac. Apple will likely announce the update during the annual Final Cut Pro Creative Summit, which begins this Wednesday. The conference is held in association with Apple, and attendees will be visiting Apple Park on the first day. Apple already teased four new features...
iPad iOS 16 WP Display Feature eric edit

Apple to Launch AI-Powered Home 'Command Center' as Soon as March 2025

Tuesday November 12, 2024 1:09 pm PST by
Apple is planning to launch an AI-powered smart home display as soon as March 2025, according to Bloomberg's Mark Gurman. The display will measure in at approximately six inches, and while it is similar to an iPad, it is square rather than rectangular and it has thick bezels around the edges. There is a camera at the top front so that it can be used for FaceTime, plus there are internal speakers...

Top Rated Comments

Apple Knowledge Navigator Avatar
20 hours ago at 05:36 am
Not quite sure what they’re expecting. It’s just glorified pattern-recognition software, and there are only so many ways to dress it up as ‘the next big thing’.
Score: 60 Votes (Like | Disagree)
wanha Avatar
20 hours ago at 05:36 am
Considering the wildly unrealistic expectations (AGI) many people have for LLMs, it's inevitable that the bubble will have to burst at some point.
Score: 57 Votes (Like | Disagree)
CarAnalogy Avatar
20 hours ago at 05:37 am

"new, untapped sources of high-quality, human-made training data"
Ha choice quote right there. They’ve scraped all they can scrape and they can’t scrape no more!

Nobody seriously thought this was going to be AGI any time soon, not anyone that was paying attention. The agent based approach in the article is the only thing they’ll be good at.

Apple should be fine. All I need Siri to do is understand the data on my phone it already has access to. I don’t need generative anything, just don’t be so dumb.
Score: 44 Votes (Like | Disagree)
iMac The Knife Avatar
20 hours ago at 06:00 am
AI is the most overblown technology since AR/VR. ChatGPT and Gemini will give different answers to the same questions a lot of the time. They will even give different answers to the same question, within each app. The amount of misinformation that is coming from AI is staggering.
Score: 24 Votes (Like | Disagree)
turbineseaplane Avatar
19 hours ago at 06:50 am
Apple upping their base RAM specs has been the only good thing from AI so far
Score: 18 Votes (Like | Disagree)
ozreth Avatar
19 hours ago at 06:44 am
Honestly **** AI. One of the biggest modern examples of a solution looking for a problem. People need to get outside, touch grass and read a book. Particularly the youth.
Score: 17 Votes (Like | Disagree)