In a lengthy blog post about Facebook's ten-year plan to "accelerate innovation and power new experiences with AI," chief technology officer Mike Schroepfer mentioned that one update coming to the company's mobile apps will be a "style transfer" tool that turns normal photos and videos into works of art using "high efficiency neural networks," all running directly on iOS and Android smartphones.

Facebook's impending update will, according to Schroepfer, run entirely on the user's smartphone and not be dependent on the content to be sent to servers, creating long load times and frustrating users. The CTO described this as the most demanding and "technically difficult" hurdle to clear in the process of adding the feature to the company's mobile apps, but Schroepfer said the company has done just that, and the result is a deep learning platform called "Caffe2Go."

facebook-art-filters

Just three months ago we set out to do something nobody else had done before: ship AI-based style transfer running live, in real time, on mobile devices. This was a major engineering challenge, as we needed to design software that could run high-powered computing operations on a device with unique resource constraints in areas like power, memory and compute capability. The result is Caffe2Go, a new deep learning platform that can capture, analyze and process pixels in real time on a mobile device.

We found that by condensing the size of the AI model used to process images and videos by 100x, we’re able to run deep neural networks with high efficiency on both iOS and Android. This is all happening in the palm of your hand, so you can apply styles to videos as you’re taking them.

Schroepfer said that the alternative of sending the content to data centers to be analyzed and filtered was "not ideal for letting people share fun content in the moment." In addition to basic image and video filtering capabilities, the new deep-learning platform will also feasibly be able to understand gesture controls when taking a selfie, for instance. In today's blog post, an example is given of a user swiping right and left between various artistic filters for a selfie (all running live, in real-time) and even snapping a picture when the user smiles.

As a comparison, the update sounds largely similar to Prisma, an app that launched over the summer and impressed many with its ability to turn photos, and eventually videos, into stylized images. Originally, Prisma used a server-side combination of neural networks and artificial intelligence to apply the different filters to user photos, but an update also introduced offline image processing, meaning users could use some of the app's filters to alter their images right from their smartphones.

For Facebook, the announcement follows a year of video-first announcements from the company, most recently embodied in CEO and co-founder Mark Zuckerberg's plan to make the camera more prominent in the app. No specific timeline was given for a possible launch of the new features on Facebook's mobile apps, but the company is clearly looking to lay the groundwork for its future, calling its new AI initiative, along with virtual reality, "new technologies that will shape the next decade."

Top Rated Comments

jmh600cbr Avatar
110 months ago
Facebook - the startup eater
Score: 3 Votes (Like | Disagree)
Zorn Avatar
110 months ago
Wow prisma filters! Sounds like a cool feature no one asked for. When will it stop looking like mud on the iPad Pro that's now a year old?
Score: 2 Votes (Like | Disagree)
-BigMac- Avatar
110 months ago
Yeh man keep beefing and slowing that sluggish app up. Why not a build in a music player?
Score: 1 Votes (Like | Disagree)
OldSchoolMacGuy Avatar
110 months ago
Prisma just added the ability to apply filters to Facebook live video on the iPhone 7 and 6S today.
Score: 1 Votes (Like | Disagree)
Hiran Avatar
110 months ago
But I deleted the official Facebook app and installed Friendly for iOS when the official app lost messenger feature. Mark Zuckerberg is dumb and arrogant as hell.
Score: 1 Votes (Like | Disagree)
smacrumon Avatar
110 months ago
Facebook is out to lunch. Their product is dated, the design is 2009 with a side of stale fries.
Score: 1 Votes (Like | Disagree)

Popular Stories

iPhone 17 Pro 34ths Perspective

iPhone 17 Pro Launching Later This Year With These 10 New Features

Sunday March 23, 2025 10:00 am PDT by
While the iPhone 17 Pro and iPhone 17 Pro Max are not expected to launch until September, there are already plenty of rumors about the devices. Below, we recap key changes rumored for the iPhone 17 Pro models as of March 2025: Aluminum frame: iPhone 17 Pro models are rumored to have an aluminum frame, whereas the iPhone 15 Pro and iPhone 16 Pro models have a titanium frame, and the iPhone ...
macbook pro blue green

When Will Apple Release the M5 MacBook Pro?

Wednesday March 26, 2025 4:53 pm PDT by
Apple regularly refreshes the MacBook Pro models, and a new version that uses M5 series chips is in the works. Apple just finished refreshing most of the Mac lineup with M4 chips, and now it's time for the M5. Rumors suggest that we could see the first M5 MacBook Pro models this fall. Design There have been no rumors of a design update for the M5 MacBook Pro models that are coming this...
Apple Lumon Terminal Pro

Apple's Mac Site Features Fictional 'Lumon Terminal Pro'

Wednesday March 26, 2025 12:19 pm PDT by
Apple is going all out with promotions for the popular Severance Apple TV+ show today, and as of right now, you'll find a new "Lumon Terminal Pro" listed on Apple's Mac site. The Lumon Terminal Pro is designed to look similar to the machines that Severance employees like Mark S. and Helly R. use for macrodata refinement. The Terminal features a blue keyboard, a small display with wide...
Facebook Feature

Facebook's New iPhone App Feature Turns the Clock Back to 2007

Thursday March 27, 2025 1:59 pm PDT by
In the mid-to-late 2000s, Facebook was all about staying connected with friends and family. However, as the social media platform added new features and grew over time, that core experience began to get drowned out. That changes starting now, according to Meta, which today introduced a new feature that will "bring back the joy" of classic Facebook. Specifically, Meta has redesigned the...
Generic iOS 18

iOS 18.4 Coming Soon With These New Features for Your iPhone

Tuesday March 25, 2025 6:45 am PDT by
Apple is expected to release iOS 18.4 to the general public as soon as next week, following more than a month of beta testing. Apple's website says some iOS 18.4 features will be released in "early April," so the update should be out as early as Tuesday, April 1. Apple this week seeded the iOS 18.4 Release Candidate, which is typically the final beta version, barring the discovery of any...
iPhone 17 Pro 34ths Perspective

iPhone 17 Pro Supports 8K Video Recording, Suggests Leaker [Updated]

Wednesday March 26, 2025 4:06 am PDT by
Update 7:25 pm: Based on comments from our forums, it appears the original Weibo post may have been mistranslated and "8K" actually refers to the high price of the device rather than 8K video recording capabilities. The iPhone 16 Pro currently starts at 7,999 yuan in China. Our original article follows below. Apple's forthcoming iPhone 17 Pro models are capable of shooting 8K video, up...
Foldable iPhone 2023 Feature 1

'iPhone Fold' to Feature Metallic Glass Hinge That Resists Deformation

Thursday March 27, 2025 4:21 am PDT by
Last week, we covered a report claiming that Apple's book-style foldable iPhone (or "iPhone Fold," as we are provisionally calling it here) will use liquid metal hinges to improve durability and help minimize screen creasing. Today, a Chinese leaker provided more details on the properties of this hinge material that help to clarify why Apple chose it for its first foldable device. According...
iOS 18 4 Ambient Music Control Center

How to Use iOS 18.4's New Ambient Music Feature in Control Center

Thursday March 27, 2025 7:45 am PDT by
The upcoming iOS 18.4 update for the iPhone adds an Ambient Music feature to Control Center. Below, we take a closer look at how it works. iOS 18.4 is currently in beta, so the Ambient Music feature is not widely available yet. The update will likely be released to the general public next week. To use the feature on iOS 18.4, open Control Center and tap on the plus sign in the top-left...
Generic iOS 19 Feature Mock

Gurman: Jon Prosser's iOS 19 Mockups 'Aren't Representative' of Redesign

Tuesday March 25, 2025 4:47 pm PDT by
The iOS 19 mockup images that leaker Jon Prosser shared today are not representative of the actual iOS 19 design, Bloomberg's Mark Gurman said on social media. According to Gurman, the images that are "floating around" are based on "very old builds" or "vague descriptions," and are lacking key features. Gurman says that we can "expect more from Apple in June." Gurman made the same comment ...