Editor’s Note: As a general version of our actionable intelligence products, this version only includes generic suggestions for brands. For industry-specific versions, please contact our Client Services Director Samantha Holland ([email protected]).
The highlights:
• Facebook stakes a claim in mobile AR with Camera Effects Platform
• Messenger Platform updated with better discovery and smarter bots
• Social VR generates new use cases for 360-degree content
On Tuesday, Facebook kicked off its annual F8 developer conference in San Jose, where the social media giant unveiled several new initiatives aimed at ushering in the next stage of its mobile empire via camera-enabled augmented reality, social VR, and new AI-powered Messenger experiences. The Lab was on the ground at this year’s event to attend developer sessions and bring you all the things that marketers need to know about Facebook’s announcements this year.
Facebook Goes All-In On Camera-Enabled Mobile AR
The biggest takeaway from this year’s F8 is that Facebook is prioritizing mobile AR in its development plan and giving Snapchat a run for its money. Facebook has been busy adding “Snapchat-clone” AR features across its mobile apps during the past year or so. However, for mobile AR to truly go mainstream, it needs to evolve beyond wacky selfie lenses and start providing real value for users. And that requires an open platform that developers can leverage to create new interactive experiences. This is where Facebook’s new Camera Effects platform comes in.
Facebook is rolling out this Camera Effects developer platform in two stages. The first one, Frame Studio, is live now and accessible to anyone with a Facebook account. You can use it to create static overlaid image frames for photos and videos shared on Facebook by simply uploading a still image, no coding required. The frames created will start appearing in the Facebook Camera for your friends or fans of a Page in May. The next stage is AR Studio, which enables designers and developers to create more sophisticated AR camera effects and interactive experiences in Facebook Camera. It is currently in closed beta, and Facebook says some AR effects will become available to public in June.
With AR Studio, designers will be able to use Photoshop-like tools to create animated masks and lenses which automatically map to facial features, such as crowns and animal faces. More AI-powered tools such as SLAM (Simultaneous Localization and Mapping) positioning and object recognition are coming later this year, which will allow developers to tie AR effects to specific objects or locations visible within the frame. In the future, users will also be able to add augmented reality notes to specific real-world locations, so they could, for example, tell friends their favorite dish at a restaurant.
The AR effects also won’t be one-size-fits-all, as developers using AR Studio will be able to customize those AR camera effects with data from the network, such as sports scores, weather, and travel data. They can also access data from the mobile device’s sensors, such as GPS location, orientation, speed, and even face/gesture recognition to create more personalized user experiences. In a clever touch, Facebook added call-to-action “Try It” buttons to appear alongside the AR effects to encourage user sampling and viral spreading. Creators will also be able to target their creations according to location, day and time, and all the standard demographic targeting from Facebook’s ad platform. For now, this targeting comes at no charge to the creator.
With the launch of this camera-based AR platform, Facebook is staking a claim in AR, building the platform and tools to create interactive experiences which use the camera as an input. Right now, that’s limited to the Facebook app on our smartphones, but Zuckerberg did mention “AR glasses” a couple of times in his opening keynote address, and Facebook will be in a good position to capitalize on glasses (or any other AR, like car windshields) with this developer platform, which allows developers to “code against the real world” and unleash a tidal wave of crowd-sourced creative to overtake Snapchat’s Lenses, which are all currently built in-house.
Messenger Platform Gets Smarter Thanks To A.I. And M Integrations
Besides the shiny new AR developer platform, Facebook is also renewing its commitment to building out the Messenger Platform with a major upgrade aimed at making its messaging app smarter and more brand-friendly. The Messenger Platform 2.0 Facebook launched on Tuesday comes with new tabs for businesses and bots, offline support for parametric QR codes, as well as new Chat Extensions that can provide new experiences in group chats.
With 200 million messages sent between businesses and users since last year and 100 thousand bots already on the platform, Facebook is already the “white pages” of the internet. Now it wants to also become the “yellow pages” with the upcoming addition of a Discover tab dedicated to chatbots, making it easier for users to discover new bots that they may find interesting or useful. Messenger users can tap the hexagon icon on the bottom right and see a list of Featured businesses along with popular bots, what’s nearby, and a variety of categories like Food & Drink, Lifestyle, and News.
To further aid bot discovery, Facebook updated the parametric QR Messenger codes to allow users to quickly connect with a bot (say at an event or a concert) just by scanning the codes with their camera in Messenger. Multiple codes can be generated for the same bot so as to offer some context such as locations and analytical data for the bot, which then may offer a more tailored response accordingly – for example, deep linking to information about a specific location, inside of a bot which covers a national retail chain. This also gives users a reason to scan even if they’re already connected to the bot.
Prior to F8, Facebook had already started to roll out its AI-powered virtual assistant M on Messenger to offer automated suggestions such as calling an Uber or looking up local restaurants based on the conversations. At this week’s event, they announced the first way for third parties to integrate with M using the new Chat Extensions functionality. Chat Extensions allow a more graphical interface to open within the message thread. For instance, Facebook announced on stage a Spotify integration that will allow people to easily search and share music with each other, without ever leaving Messenger. They work in a similar way as Apple’s iMessage apps, where you can pull up an applet to access a developer-controlled experience within Messenger, and then share the results of that back to your chat. Over time, M will learn to recognize a task a user is trying to accomplish, and recommend Chat Extensions which may be able to help.
Facebook is also working to make its messaging system smarter and more responsive by launching Smart Replies for small businesses that don’t necessarily have the resources to build a full-on chatbot but still want an easy way to communicate with customers. This new AI-enabled feature will allow businesses to enable automated answers to frequently asked questions, thus simplifying the customer service on Facebook Pages and Messenger. Smart Replies are currently being trialed with U.S. restaurants, with information populated directly from their websites, no data entry or setup required.
Social VR Generates New Use Case For 360-Degree Content
Looking beyond the camera and Messenger, Facebook is also taking a big step in virtual reality with the launch of Facebook Spaces, marking the company’s first significant integration of virtual reality into its core social product. It will run on Oculus Rift with the Touch controllers to bring social interactions into 3D virtual spaces, allowing users to generate personalized avatars and hang out together in virtual reality.
Unlike most online social platforms, the communication in Facebook Spaces won’t be text-oriented, but instead will rely on voice, facial expressions, and body language. In a demo, Facebook’s photo recognition AI helps users to create animated avatars of themselves, who can then hang out together in both virtual spaces as well as real world settings based on 360-degree photos and videos. Facebook describes it as a “virtual dinner party,” where users can talk, create 3D objects, explore 360 videos, and even take in-VR selfies.
Facebook has indicated that although Spaces is designed to be a home screen for VR, third party integrations won’t appear in the forms of VR apps, but rather as environments and objects inside the virtual space.
In addition, Facebook Spaces can use Messenger to place video calls to people out in real world, which is an interesting distribution strategy to spread awareness for this new product. People who receive video calls from Facebook Spaces will show up in the virtual space on a flat screen as normal video calls would appear, but they will be able to see and talk with their friends’ virtual avatars.
What Brands Need To Do
Altogether, what Facebook announced on Tuesday are indicative of how the world’s biggest online audience aggregator is going to evolve, and it is up to brands to keep up with this evolution into mobile AR and social VR. The launch of the Camera Effects platform holds great potential for brands, as it allows them to work with developers to create AR camera effects to engage mobile users and, in some cases, pull in personal and contextual data to create unique experiences that they will want to share.
Facebook showcased some AR effects from its brand partners on stage, including a Nike headband overlay that featured bragging right stats from your recent run and an overlaid map, a Mass Effect-themed helmet mask that pulls in stats from your game progress, as well as a video lens created by Manchester United that pulls in real-time soccer match scores, completed with “GOAL!” and falling confetti overlays.
For now, however, there are currently no paid placement opportunities. The AR effects will be surfaced via a ranking algorithm within the camera itself, based on attributes such as location, date/time, and keywords, as well as how likely they think an individual is to share according to their past behaviors. But the “Try It” call-to-actions on shared media is an incentive to create great lenses, as it will help them earn organic distribution on the platform.
Brands looking to stay ahead of the technological curve need to start developing a AR strategy today to organically insert their brands directly into the camera and therefore into the photos and videos that people share on Facebook. While the current lenses appear similar to Snapchat, the open developer platform means that Facebook’s AR lenses will advance quickly, and are available to anyone with the designers and developers to create compelling experiences.
For CPG brands, this could mean creating fun, seasonal frames or AR lenses to remind consumers of your products and target users based on Facebook’s data, whereas healthcare and fitness brands can take some inspiration from the Nike headband lens Facebook demoed and start thinking about how to create similar AR camera effects that can amplify and enrich a great health and fitness experience for their customers.
With Messenger Platform 2.0., brands will have better tools to create smarter chatbots to connect with the over 1.2 billion active users on Messenger. Brands should consider creating various physical call-to-actions with the new parametric QR Messenger codes linking to their chatbots within different contexts, thus creating convenient chat experiences for users by deep-link them to specific functions within the bots. For restaurants and QSR brands, this could creating different QR codes for different restaurant locations and leverage location data to provide a localized experience for the customers when they turn to chatbots for information or directions.
In addition, brands also need to start thinking about ways to use Chat Extension to create experiences within Messenger that don’t fit in a bot/conversational interface. For retailers, this could mean creating an extension similar to what Mode.ai is doing to reach shoppers in chats.
While the beta launch of Facebook Spaces is not integrated with Facebook’s main platform and therefore limited in its opportunities for brand integrations, this is only the starting point for social VR, and we are confident that brands will have plenty of ways to get into the virtual social spaces as Facebook continues to build out this platform.
As many of the virtual environments in Facebook Spaces are generated from 360-degree photos and videos, pulled right from a user’s Facebook Photos, it’s a great incentive for brands to start investing in 360 content now, and getting it in the hands of their customers. Sharing that content in Spaces could be a great way to enable existing fans to advocate for your destination, hotel, car, or entertainment experience.
How We Can Help
While mobile AR and social VR are still in early stages of development, brands greatly benefit by starting to develop strategies for these two emerging areas. If you’re not sure where to start, the Lab is at your service.
The Lab has always been fascinated by the enormous potential of AR and its ability to transform our physical world. If you haven’t read our analysis on the Pokémon Go phenomenon last summer, which introduced AR technology to mainstream consumers, you can check it out here for more AR-related brand suggestions. We’re excited at having an open development platform from Facebook, allowing us to develop mobile AR experiences that can scale to reach billions of people. If you’d like to discuss more about how your brand can properly harness the power of mobile AR to engage your customers and create extra value, please reach out and get in touch with us.
We also have extensive experience in building AI-powered chatbots to reach consumers on messaging apps. The Niro bot we built in collaboration with Ansible for Kia is a good example of our chatbot expertise. To help our clients create an optimized conversational experience, we developed Dialogue, a dedicated conversational practice backed by our stack of best-in-class technology partners and an insights engine that extracts business intelligence from conversational data.
As for VR, our dedicated team of experts is here to guide marketers through the distribution landscape. We work closely with brands to develop sustainable VR content strategies to promote branded VR and 360 video content across various apps and platforms. With our proprietary technology stack powered by a combination of best-in-class VR partners, we offer customized solutions for distributing and measuring branded VR content that truly enhance brand messaging and contribute to the campaign objectives.
If you’d like to know how the Lab can help your brand figure out how to tap into these tech trends manifested at F8 to supercharge your marketing efforts, please contact our Client Services Director Samantha Holland ([email protected]) to schedule a visit to the Lab.
