In a lengthy blog post about Facebook's ten-year plan to "accelerate innovation and power new experiences with AI," chief technology officer Mike Schroepfer mentioned that one update coming to the company's mobile apps will be a "style transfer" tool that turns normal photos and videos into works of art using "high efficiency neural networks," all running directly on iOS and Android smartphones.

Facebook's impending update will, according to Schroepfer, run entirely on the user's smartphone and not be dependent on the content to be sent to servers, creating long load times and frustrating users. The CTO described this as the most demanding and "technically difficult" hurdle to clear in the process of adding the feature to the company's mobile apps, but Schroepfer said the company has done just that, and the result is a deep learning platform called "Caffe2Go."

facebook-art-filters

Just three months ago we set out to do something nobody else had done before: ship AI-based style transfer running live, in real time, on mobile devices. This was a major engineering challenge, as we needed to design software that could run high-powered computing operations on a device with unique resource constraints in areas like power, memory and compute capability. The result is Caffe2Go, a new deep learning platform that can capture, analyze and process pixels in real time on a mobile device.

We found that by condensing the size of the AI model used to process images and videos by 100x, we’re able to run deep neural networks with high efficiency on both iOS and Android. This is all happening in the palm of your hand, so you can apply styles to videos as you’re taking them.

Schroepfer said that the alternative of sending the content to data centers to be analyzed and filtered was "not ideal for letting people share fun content in the moment." In addition to basic image and video filtering capabilities, the new deep-learning platform will also feasibly be able to understand gesture controls when taking a selfie, for instance. In today's blog post, an example is given of a user swiping right and left between various artistic filters for a selfie (all running live, in real-time) and even snapping a picture when the user smiles.

As a comparison, the update sounds largely similar to Prisma, an app that launched over the summer and impressed many with its ability to turn photos, and eventually videos, into stylized images. Originally, Prisma used a server-side combination of neural networks and artificial intelligence to apply the different filters to user photos, but an update also introduced offline image processing, meaning users could use some of the app's filters to alter their images right from their smartphones.

For Facebook, the announcement follows a year of video-first announcements from the company, most recently embodied in CEO and co-founder Mark Zuckerberg's plan to make the camera more prominent in the app. No specific timeline was given for a possible launch of the new features on Facebook's mobile apps, but the company is clearly looking to lay the groundwork for its future, calling its new AI initiative, along with virtual reality, "new technologies that will shape the next decade."

Top Rated Comments

jmh600cbr Avatar
116 months ago
Facebook - the startup eater
Score: 3 Votes (Like | Disagree)
Zorn Avatar
116 months ago
Wow prisma filters! Sounds like a cool feature no one asked for. When will it stop looking like mud on the iPad Pro that's now a year old?
Score: 2 Votes (Like | Disagree)
-BigMac- Avatar
116 months ago
Yeh man keep beefing and slowing that sluggish app up. Why not a build in a music player?
Score: 1 Votes (Like | Disagree)
OldSchoolMacGuy Avatar
116 months ago
Prisma just added the ability to apply filters to Facebook live video on the iPhone 7 and 6S today.
Score: 1 Votes (Like | Disagree)
Hiran Avatar
116 months ago
But I deleted the official Facebook app and installed Friendly for iOS when the official app lost messenger feature. Mark Zuckerberg is dumb and arrogant as hell.
Score: 1 Votes (Like | Disagree)
smacrumon Avatar
116 months ago
Facebook is out to lunch. Their product is dated, the design is 2009 with a side of stale fries.
Score: 1 Votes (Like | Disagree)

Popular Stories

Tim Cook Rainbow

Apple Event in October? Here's What to Expect

Monday September 29, 2025 9:31 am PDT by
Apple's annual iPhone event is in the rearview mirror, but rumors suggest the company plans to release a handful of additional products before the year ends. Will there be another Apple event this October? We discuss the possibility below. Apple in October Apple's most recent October events were in 2021 and 2023. In 2022 and 2024, Apple did not host an October event. Instead, it...
apple wallet drivers license feature iPhone 15 pro teal 1

Apple's iPhone Driver's License Feature Now Available in 11 U.S. States

Tuesday September 30, 2025 6:40 am PDT by
In select U.S. states, residents can add their driver's license or state ID to the Wallet app on the iPhone and Apple Watch, providing a convenient and contactless way to display proof of identity or age at select airports and businesses, and in select apps. Apple recently revealed that the feature would soon be available in North Dakota, and starting today, the feature has officially gone...
space black mbp

Here's Every New Apple Product That Leaked Yesterday

Wednesday October 1, 2025 8:27 am PDT by
A handful of upcoming Apple products leaked yesterday, through a combination of YouTube videos out of Russia and U.S. Federal Communications Commission (FCC) documents that were released, despite Apple's confidentiality requests. The leaked products include an iPad Pro with an M5 chip, as well as updated MacBook Pro and Apple Vision Pro models. All of these devices had already been rumored...
maxresdefault

New iPad Pro With M5 Chip Leaked in Unboxing Video

Tuesday September 30, 2025 8:39 am PDT by
An apparent unboxing video for an unannounced iPad Pro with the M5 chip was uploaded to YouTube today by Russian channel Wylsacom. The same YouTube account leaked the 14-inch MacBook Pro with the M4 chip before it was announced by Apple last year, so this is likely a legitimate leak. Based on the box shown in the video, this appears to be a 13-inch iPad Pro with an M5 chip, 256GB of...
fcc vision pro leak

FCC Accidentally Leaks Apple's Next Vision Pro

Tuesday September 30, 2025 3:48 pm PDT by
The United States Federal Communications Commission (FCC) has seemingly confirmed Apple's work on an updated version of the Vision Pro headset. One of several documents the FCC shared today references an Apple-designed "Head Mounted Device" with a model number of A3416. An included image confirms the device is a Vision Pro. The FCC's uploads are transmission tests, SAR test reports, and...
macbook pro prime day 2025

FCC Leaks Upcoming MacBook Pro and More

Tuesday September 30, 2025 1:58 pm PDT by
The United States Federal Communications Commission has confirmed Apple's work on a new version of the MacBook Pro and several other products, leaking details on the devices ahead of launch. The FCC published documents that reference model numbers that do not correspond with existing devices. A3434, for example, references an unreleased MacBook Pro, while other numbers are likely for...
iOS 26 Feature

iOS 26.1 to iOS 26.4 Will Add These New Features to Your iPhone

Wednesday October 1, 2025 1:26 pm PDT by
iOS 26 was released last month, but the software train never stops, and iOS 26.1 beta testing is already underway. So far, iOS 26.1 makes both Apple Intelligence and Live Translation on compatible AirPods available in additional languages, and it includes some other minor changes across Apple Music, Calendar, Photos, and Safari. More features and changes will follow in future versions,...
iOS 26

Apple Releases iOS 26.0.1 With Fixes for Wi-Fi, Cellular, and Camera Issues on iPhone 17 Models

Monday September 29, 2025 10:12 am PDT by
Apple today released iOS 26.0.1 and iPadOS 26.0.1, the first updates to the iOS 26 and iPadOS 26 operating systems that came out earlier this week. The new software can be downloaded on eligible iPhones and iPads over-the-air by going to Settings > General > Software Update. According to Apple's release notes for the update, iOS 26.0.1 addresses a bug that could cause aberrations in...
Apple 2025 Thumb 1

Apple's 2025 Product Roadmap: What's Still Coming

Wednesday October 1, 2025 3:56 pm PDT by
Apple's two big yearly events, WWDC and the iPhone launch, are done and over with, but there are still some new products that we're expecting to see before the end of the year. Apple TV The Apple TV hasn't been updated since 2022, so it's due for a refresh. It doesn't look like Apple is going to change the design of its set-top box, but we can expect a faster chip Apple code suggests...
iOS 26 Everything New Feature

26 Hidden iOS 26 Tricks to Change How You Use Your iPhone

Wednesday October 1, 2025 9:16 am PDT by
Apple released iOS 26 in mid-September, bringing a range of new features and changes to iPhones across the globe. But not all of the included improvements have been showcased with Apple's typical fanfare, and many are likely to have been overlooked. Below, we've highlighted 26 lesser known additions and enhancements that could potentially change how you use your iPhone on a daily basis. Got...