In a lengthy blog post about Facebook's ten-year plan to "accelerate innovation and power new experiences with AI," chief technology officer Mike Schroepfer mentioned that one update coming to the company's mobile apps will be a "style transfer" tool that turns normal photos and videos into works of art using "high efficiency neural networks," all running directly on iOS and Android smartphones.

Facebook's impending update will, according to Schroepfer, run entirely on the user's smartphone and not be dependent on the content to be sent to servers, creating long load times and frustrating users. The CTO described this as the most demanding and "technically difficult" hurdle to clear in the process of adding the feature to the company's mobile apps, but Schroepfer said the company has done just that, and the result is a deep learning platform called "Caffe2Go."

facebook-art-filters

Just three months ago we set out to do something nobody else had done before: ship AI-based style transfer running live, in real time, on mobile devices. This was a major engineering challenge, as we needed to design software that could run high-powered computing operations on a device with unique resource constraints in areas like power, memory and compute capability. The result is Caffe2Go, a new deep learning platform that can capture, analyze and process pixels in real time on a mobile device.

We found that by condensing the size of the AI model used to process images and videos by 100x, we’re able to run deep neural networks with high efficiency on both iOS and Android. This is all happening in the palm of your hand, so you can apply styles to videos as you’re taking them.

Schroepfer said that the alternative of sending the content to data centers to be analyzed and filtered was "not ideal for letting people share fun content in the moment." In addition to basic image and video filtering capabilities, the new deep-learning platform will also feasibly be able to understand gesture controls when taking a selfie, for instance. In today's blog post, an example is given of a user swiping right and left between various artistic filters for a selfie (all running live, in real-time) and even snapping a picture when the user smiles.

As a comparison, the update sounds largely similar to Prisma, an app that launched over the summer and impressed many with its ability to turn photos, and eventually videos, into stylized images. Originally, Prisma used a server-side combination of neural networks and artificial intelligence to apply the different filters to user photos, but an update also introduced offline image processing, meaning users could use some of the app's filters to alter their images right from their smartphones.

For Facebook, the announcement follows a year of video-first announcements from the company, most recently embodied in CEO and co-founder Mark Zuckerberg's plan to make the camera more prominent in the app. No specific timeline was given for a possible launch of the new features on Facebook's mobile apps, but the company is clearly looking to lay the groundwork for its future, calling its new AI initiative, along with virtual reality, "new technologies that will shape the next decade."

Top Rated Comments

jmh600cbr Avatar
116 months ago
Facebook - the startup eater
Score: 3 Votes (Like | Disagree)
Zorn Avatar
116 months ago
Wow prisma filters! Sounds like a cool feature no one asked for. When will it stop looking like mud on the iPad Pro that's now a year old?
Score: 2 Votes (Like | Disagree)
-BigMac- Avatar
116 months ago
Yeh man keep beefing and slowing that sluggish app up. Why not a build in a music player?
Score: 1 Votes (Like | Disagree)
OldSchoolMacGuy Avatar
116 months ago
Prisma just added the ability to apply filters to Facebook live video on the iPhone 7 and 6S today.
Score: 1 Votes (Like | Disagree)
Hiran Avatar
116 months ago
But I deleted the official Facebook app and installed Friendly for iOS when the official app lost messenger feature. Mark Zuckerberg is dumb and arrogant as hell.
Score: 1 Votes (Like | Disagree)
smacrumon Avatar
116 months ago
Facebook is out to lunch. Their product is dated, the design is 2009 with a side of stale fries.
Score: 1 Votes (Like | Disagree)

Popular Stories

iOS 26 on iPhone Feature

Here's When iOS 26 Rolls Out Today in Every Time Zone [Update: Out Now!]

Monday September 15, 2025 12:00 am PDT by
Update 10:06 a.m.: iOS 26 is rolling out now, though it may take a bit for all users to see it, so keep checking! Today's the day! Apple is about to release iOS 26, which will deliver the biggest redesign since iOS 7 and bring a range of new features and improvements to iPhones worldwide. It's Apple's biggest software update of the year, and Apple announced at last week's iPhone event that...
Tim Cook Rainbow

Apple Reportedly Plans to Launch These 10 Products in 'Coming Months'

Sunday September 14, 2025 8:45 am PDT by
Apple's annual September event is now in the rearview mirror, with the iPhone 17, iPhone 17 Pro, iPhone 17 Pro Max, iPhone Air, Apple Watch Series 11, Apple Watch Ultra 3, Apple Watch SE 3, and AirPods Pro 3 set to launch this Friday, September 19. As always, there is more to come. In his Power On newsletter today, Bloomberg's Mark Gurman said Apple plans to release many products in the...
iOS 26 Battery Glass Feature

Apple Says Installing iOS 26 Might Impact Battery Life

Monday September 15, 2025 10:56 am PDT by
In the iOS 26 release notes, Apple is warning iPhone users that installing the new software might have a temporary impact on battery life, which is normal. A new support document explains that major iOS updates require background setup like indexing data and files for search, downloading new assets, and updating apps. Further, Apple says that new features could require more resources,...
apple n1 chip

Apple's New N1 Chip in iPhone 17, iPhone 17 Pro, and iPhone Air Has a Wi-Fi 7 Limitation

Saturday September 13, 2025 10:01 am PDT by
The latest iPhone 17, iPhone 17 Pro, iPhone 17 Pro Max, and iPhone Air models are equipped with Apple's all-new N1 chip for Wi-Fi 7, Bluetooth 6, and Thread connectivity. However, the chip has a Wi-Fi 7 bandwidth limitation. According to FCC documents reviewed by MacRumors, the N1 chip in all of the new iPhone models supports up to 160 MHz channel bandwidth for Wi-Fi 7, short of the...
iphone 17 lineup

iPhone 17 Models Launch on September 19 With These New Features

Friday September 12, 2025 7:58 am PDT by
Apple will launch its new iPhone 17 lineup and ultra-thin iPhone Air in stores on Friday, September 19, and the company has already shown off the new devices at its fall event, which ran with the the tagline "Awe dropping." The iPhone 17 series brings a host of new features and enhancements. Here's a rundown of the biggest upgrades and changes: iPhone 17 Display Changes The iPhone...
AirPods Pro Firmware Feature

AirPods Pro 2 and AirPods 4 Get iOS 26 Features With New Firmware Update

Monday September 15, 2025 10:50 am PDT by
Apple today released updated firmware for the AirPods Pro 2 and the AirPods 4, introducing support for the new AirPods features that are included in iOS 26, iPadOS 26, and macOS Tahoe. The firmware has a build number of 8A356, and it replaces the current 7E93 firmware. With Apple's new software updates, the AirPods Pro 2 and the AirPods 4 support better audio quality for phone calls and...
iPhone 17 Pro Air Boxes

iPhone Air and iPhone 17 Pro Boxes Revealed

Sunday September 14, 2025 1:36 pm PDT by
T-Mobile President Jon Freier today shared real-world photos of Apple's boxes for the iPhone Air, iPhone 17 Pro, and iPhone 17 models, which launch on Friday. Image Credit: Jon Freier Apple has typically included iPhone box renders in its product environmental reports, but it did not do so for the latest models. However, Apple's iPhone Upgrade Program page does offer some images of the boxes, ...