Apple Develops Breakthrough Method for Running LLMs on iPhones

Apple GPT in your pocket? It could be a reality sooner than you think. Apple AI researchers say they have made a key breakthrough in deploying large language models (LLMs) on iPhones and other Apple devices with limited memory by inventing an innovative flash memory utilization technique.

siri symbol iphone

LLMs and Memory Constraints

LLM-based chatbots like ChatGPT and Claude are incredibly data and memory-intensive, typically requiring vast amounts of memory to function, which is a challenge for devices like iPhones that have limited memory capacity. To tackle this issue, Apple researchers have developed a novel technique that uses flash memory – the same memory where your apps and photos live – to store the AI model's data.

Storing AI on Flash Memory

In a new research paper titled "LLM in a flash: Efficient Large Language Model Inference with Limited Memory," the authors note that flash storage is more abundant in mobile devices than the RAM traditionally used for running LLMs. Their method cleverly bypasses the limitation using two key techniques that minimize data transfer and maximize flash memory throughput:

  1. Windowing: Think of this as a recycling method. Instead of loading new data every time, the AI model reuses some of the data it already processed. This reduces the need for constant memory fetching, making the process faster and smoother.
  2. Row-Column Bundling: This technique is like reading a book in larger chunks instead of one word at a time. By grouping data more efficiently, it can be read faster from the flash memory, speeding up the AI's ability to understand and generate language.

The combination of these methods allows AI models to run up to twice the size of the iPhone's available memory, according to the paper. This translates to a 4-5 times increase in speed on standard processors (CPUs) and an impressive 20-25 times faster on graphics processors (GPUs). "This breakthrough is particularly crucial for deploying advanced LLMs in resource-limited environments, thereby expanding their applicability and accessibility," write the authors.

Faster AI on iPhone

The breakthrough in AI efficiency opens new possibilities for future iPhones, such as more advanced Siri capabilities, real-time language translation, and sophisticated AI-driven features in photography and augmented reality. The technology also sets the stage for iPhones to run complex AI assistants and chatbots on-device, something Apple is already said to be working on.

Apple's work on generative AI could eventually be incorporated into its ‌Siri‌ voice assistant. Apple in February 2023 held an AI summit and briefed employees on its large language model work. According to Bloomberg, Apple is aiming for a smarter version of Siri that's deeply integrated with AI. Apple is planning to update the way that ‌Siri‌ interacts with the Messages app, allowing users to field complex questions and auto-complete sentences more effectively. Beyond that, Apple is rumored to be planning to add AI to as many Apple apps as possible.

Apple GPT

Apple is reportedly developing its own generative AI model called "Ajax". Designed to rival the likes of OpenAI's GPT-3 and GPT-4, Ajax operates on 200 billion parameters, suggesting a high level of complexity and capability in language understanding and generation. Internally known as "Apple GPT," Ajax aims to unify machine learning development across Apple, suggesting a broader strategy to integrate AI more deeply into Apple's ecosystem.

As of the latest reports, Ajax is considered more capable than the earlier generation ChatGPT 3.5. However, it's also suggested that OpenAI's newer models may have advanced beyond Ajax's capabilities as of September 2023​.

Both The Information and analyst Jeff Pu claim that Apple will have some kind of generative AI feature available on the ‌iPhone‌ and iPad around late 2024, which is when iOS 18 will be coming out. Pu said in October that Apple is building a few hundred AI servers in 2023, with more to come in 2024. Apple will reportedly offer a combination of cloud-based AI and AI with on-device processing.

Popular Stories

iOS 18

Apple Says iOS 18.4 Will Be Released in April With These New Features

Wednesday February 26, 2025 7:15 am PST by
In a recent press release, Apple confirmed that iOS 18.4 will be released in April. From the Apple News+ Food announcement:Coming with iOS 18.4 and iPadOS 18.4 in April, Apple News+ subscribers will have access to Apple News+ Food, a new section that will feature tens of thousands of recipes — as well as stories about restaurants, healthy eating, kitchen essentials, and more — from the...
iphone 16e usb c feature

Apple Provides Reason for iPhone 16e's Lack of MagSafe

Friday February 28, 2025 4:39 am PST by
Apple has offered a reason why the iPhone 16e doesn't include MagSafe, one of the more notable omissions from its latest entry-level smartphone. According to Apple representatives who spoke to Daring Fireball's John Gruber, MagSafe is not included in the iPhone 16e because "most people in the iPhone 16e's target audience exclusively charge their phones by plugging them into a charging...
Generic iPhone 17 Feature With Full Width Dynamic Island

Latest iPhone 17 Series CAD Images in Line With Redesign Rumors

Friday February 28, 2025 2:51 am PST by
Apple is expected to embrace a new camera system design for some models in its upcoming iPhone 17 series, and the latest purported CAD images don't deviate from what we have been hearing lately about Apple's new lineup. If you do not like the sound of an iPhone with a Google Pixel-style camera bar, look away now. Seasoned leaker Sonny Dickson shared the following images in a post on X...
cook trump

Trump Responds to Apple Keeping Diversity Policies

Wednesday February 26, 2025 6:32 am PST by
In an all-caps post on Truth Social today, U.S. President Donald Trump said Apple should fully end its diversity, equity, and inclusion (DEI) policies. Tim Cook meeting with President Trump in 2017 "APPLE SHOULD GET RID OF DEI RULES, NOT JUST MAKE ADJUSTMENTS TO THEM," he wrote. Trump's post comes one day after Apple held its annual shareholders meeting, during which a majority of...
apple intelligence black

These New Apple Intelligence Features Are Coming in iOS 18.4

Friday February 28, 2025 3:17 pm PST by
iOS 18.4 was supposed to bring new Apple Intelligence Siri features, but Apple ended up needing to pull those capabilities from the update to continue testing. There are fewer new Apple Intelligence additions now, but there are still some new features that will make the update worth installing when it comes out in April. Priority Notifications Apple introduced Priority Notifications back at ...
apple watch ultra snow

6 Features Coming to the Apple Watch Ultra 3

Tuesday February 25, 2025 9:00 am PST by
The Apple Watch Ultra 3 is expected to launch later this year, arriving two years after the previous model with a series of improvements. While no noticeable design changes are expected for the third generation since the company tends to stick with the same Apple Watch design through three generations before changing it, there are a series of internal upgrades on the way. By the time the ...
airpods pro purple

Here's When AirPods Pro 3 Are Rumored to Launch

Monday February 24, 2025 9:14 am PST by
According to a post on X today from a leaker known as Kosutami, Apple plans to launch AirPods Pro 3 in May or June this year. The leaker also claimed that an AirTag 2 will launch around the same time. Kosutami is best known as a collector of prototype Apple hardware, but they have occasionally shared accurate information about Apple's future product plans. For example, they accurately...
apple c1

How Fast is Apple's First-Ever 5G Modem? The Results Are Surprising

Friday February 28, 2025 10:08 am PST by
iPhone 16e reviews are now out, and Apple's custom-designed C1 modem has been put to the test. The results so far are quite surprising, as the C1's speeds are not as slow compared to Qualcomm modems as originally expected. While the C1 does not support ultra-fast mmWave 5G in the U.S., it appears to offer comparable 5G performance to Qualcomm's Snapdragon X71 modem found in the iPhone 16,...

Top Rated Comments

grantishere Avatar
16 months ago
“Siri, when’s Apple GPT going to be released?”

“Here are the neatest pizza restaurants. Would you like directions?”
Score: 66 Votes (Like | Disagree)
Infodataset Avatar
16 months ago
Apple is so clever to avoid adding more ram ?
Score: 54 Votes (Like | Disagree)
Abazigal Avatar
16 months ago

Does this mean Siri will finally be able to “call my wife on iPhone” again after losing that ability around 2-3 years ago?
At least it didn't respond with "which one?" ?
Score: 34 Votes (Like | Disagree)
wdfly Avatar
16 months ago
LOL innovative invention of swapping memory to storage…… maybe they can call it something cool like “cache”.
Score: 23 Votes (Like | Disagree)
subjonas Avatar
16 months ago
Sounds like some significant headway. I’d definitely like as much AI done on device as possible.

Apple will probably always be behind with their LLM as long as they prioritize privacy, which I’m very ok with. But just like with big phones, they will bend if there is enough market pressure, which I suspect may eventually be the case.
Score: 21 Votes (Like | Disagree)
contacos Avatar
16 months ago
I'll be impressed when Siri is finally multi-lingual. I cannot believe Siri was released with iPhone 4S and it still cannot do it. It could be as simple as being "aware" what language you usually type in with contact A and automatically use Siri in language X to communicate with this person while using language Y with contact Y. All they did was "this text is in another language, do you still want me to read [the gibberish nonsense]" with iOS 16
Score: 18 Votes (Like | Disagree)