Snapchat, the ephemeral photo and video messenger from Snap, has come a long way since it was initially released in 2011. Over the years, the company has expanded its functionality to include new features powered by augmented reality (AR), leveraging its home-grown wearable hardware. While Snap is far from the only firm working on incorporating AR technology into its products — we recently saw Oppo and Xiaomi unveil wearable AR hardware at the Mobile World Congress (MWC) 2023 — the company is also actively working on helping creators monetise their content by working with brands. Gadgets 360 sat down with Joe Darko, Global Head of AR Developer Relations at Snap, to discuss the company’s plans for AR content, the creator economy in India, and future AR-related hardware offerings.
Gadgets 360: Tell us about Snapchat’s relationship with AR and how Snap has increased the use of AR in the app?
Joe Darko: Eight years ago, we launched Lenses, and they actually help drive engagement on Snapchat. When you think about it, people are using rainbow Lenses and dog-ear Lenses to really engage with their audience and communicate with people. So over time that has evolved, right, we see that Lenses have become more and more innovative. About 250 million Snapchatters use AR on a daily basis, and that is a lot. These experiences are created by about 300,000 AR creators and developers around the world. These people bring their skill-sets to the Lens Studio platform and create Lenses which are used by Snapchat users in stories, messaging and even on Spotlight.
So that is how Snapchat has been able to grow as well. When Lenses started out, it was more about fun and entertainment, but it’s now moving more towards innovative use cases like commerce and education. That evolution has actually enabled us to really bring in more advanced AR creators and AR developers who are leveraging the latest technology to really build advanced experiences. Think of multiplayer experiences with connected Lenses. So, you and I can actually play a game within the Lens itself.
Recently, we launched ray tracing technology; the first of its kind on mobile. Now, you can see a Lens become more real due to light estimation or reflecting of light on digital objects. This technology could bring to life AR experiences in Shopping for example. Lenses that feature jewellery could look real with this technology.
Gadgets 360: Can a user who is unable to write code create Lenses on their own? Can you explain the process of developing AR Lenses for non-technical users?
Joe Darko: Anyone can create a Lens; we have people from artistic and design backgrounds, to hardcore developers who use advanced code to build AI experiences. Even in our ecosystem, with the programs that we run, we try to cater to both ends of the spectrum.
We also have another tier — partners. These are agency studios or tiger teams who work with individual creators and developers to build AR experiences. You can open Lens Studio right now and create your first Lens by leveraging our templates. And as you continue to become more competent, you’re going to grow in skill sets and leverage more advanced features to build more innovative Lenses.
Gadgets 360: How is Snapchat evolving amid increasing AR adoption? Does AR build on the original idea of disappearing messages or does this broaden its scope to completely different use cases?
Joe Darko: When you look at Snapchat, you’re talking about the ephemeral aspect of the app, where once someone sees a message, it’s gone, right? That ephemerality is still there. It is a core piece of the app and human communication. But people are also using AR to communicate and share their experiences. It’s made communication more fun and more engaging, because it’s not just sending a text message. But when we look at AR, we’re thinking about it from a perspective like commerce.
People are now building these AR experiences from a business perspective; from a commerce perspective. For instance, fashion brands are actually showing what AR can do. You can see how clothes look on you before you try them on. Aside from just chatting, you can also engage with friends or other Snapchatters in AR games.
Lens Studio also enables you to build and publish your Lens outside of Snapchatusing our developer SDKsoftware, Camera Kit. Let’s say you have a dating app with CameraKit, you will be able to leverage the camera functionality of Snapchat into your app. This means your dating app users could now have Lenses available to them for use to communicate and interact in an immersive and a more engaging way. Interestingly, over 250 million engage with augmented reality on Snapchat every day. That is slightly higher than the total mDAU of Twitter last quarter (237.8 million). Snapchatters play with AR Lenses 6B+ times per day on average. CameraKit opens up this opportunity to more and more users.
Gadgets 360: How important are Lenses to Snapchat’s creator community? Can you share some numbers on how many users in India actively use Lenses on Snapchat?
Joe Darko: So, I won’t be able to go deep into specifics about the data at the moment, but I can start by talking about Lens creators and developers in the country. Snap’s Indian AR creator community, who are also active users of Lens Studio – Snap’s free desktop application designed to help artists, designers, and developers build AR experiences – grew by 60 percent in 2022.
We’re also working with the AR ecosystem of creators, developers and partners to build locally relevant Lenses. Last year, we had about 75 festivals for the year. Lenses are not only for festivals, they’re also for moments like Valentine’s Day; they can also be city-specific, like so Indore has a geofilter for that particular specific day for that city. These are the moments that we’re building for.
And we are always tapping into our AR creator ecosystem to build some of these experiences. The community continues to grow with more and more students and developers expressing their interest by attending in-person meetups across India. Since January, we’ve had about 20 meetups in multiple cities including Bangalore, Mumbai, Delhi, and Hyderabad. We’ve reached about 1,000 developers at these meetups. Creators who are interested in learning about Lens Studio and building Lenses by leveraging the platform attend these events. We recently had one where approximately 200 Lenses were submitted by about 180 students.
Gadgets 360: What are specific use cases of AR for interpersonal messaging? When and how would the average person use AR in their communications?
Joe Darko: I feel like AR ensures that my communications are more enriched. Communication has to have some level of substance. For instance, my family is a perfect example. I’m not good with home decorations and design, so I can communicate with my wife — when I’m traveling, and through AR and Lenses, get a better sense of what she is trying to convey. If we were to be in the market for a new couch, she could connect with me and say “Look, that’s how this couch is going to look in our living room or loft area”. And also, it keeps conversations entertaining instead of being monotonous. My wife uses Snapchat to send me messages with Lenses on our kids’ faces, instead of just basic pictures of them.
AR Lenses make communication richer, and get more people engaged and involved in communication while adding a lot of colour to the conversation. Sometimes when we’re not talking face to face, there’s a lot of elements missing. With AR, we are able to add different elements of communication to feel more engaged; more involved.
Think about it this way – AR is more immersive. And so if your communication is immersive, you really get in-depth with people, they’re drawn into whatever you’re talking about. And we believe that AR is the most compelling path forward, when we talk about the next reality or the XR industry. We have actually dedicated ourselves to AR because we think it enriches our lives as we’re more immersed in it in a very healthy way. We’re not transported it into a whole new virtual world, we’re still involved in our surroundings — we’re still present and engaged.
Gadgets 360: Snapchat users are also creating AR Lenses that are used by brands for marketing purposes. Can you share some examples of creators in India that have monetised their Snapchat AR Lenses?
Joe Darko: Yeah, the first bit I’ll touch on is actually one of the most important things here — monetisation. If you want creators and developers in the ecosystem to build a platform, there has to be a path for them to monetise. And these branded opportunities give developers a chance to really monetise, and continue to be embedded in a platform and grow their skill sets. We’re able to connect our creators to brands and businesses who want AR experiences, and it gives developers a chance to create, make money and build businesses.
There’s a very robust ecosystem in India and a lot of our creators have started businesses. They started off as individual creators or developers, and then they partnered with others and started a business and now they build for brands and businesses who add monetary value to the ecosystem.
Take the example of Gayathri and Pradeepa, two talented and visionary women who started their journey as AR app developers and quickly evolved into social AR developers. Today, they are successful Snapchat AR experience creators and have founded their own private studio. They started with three people and have now expanded to more than 15 full-time employees along with eight freelancing consultants — these include developers, designers and modelers. And this is a business they started about a year and a half ago.
They started by developing simple AR apps and gradually worked their way up to creating complex AR experiences for major brands. Their company has created over 150 AR experiences for some of the biggest names in the industry, including Google Lens, Pepsi, Disney+ Hotstar, Vijay Television, Zee Entertainment, Neutrogena, Flipkart and many more.
Meanwhile, 24-year-old Kavin Kumar started social AR in 2018 as a hobby and worked for a creative studio making social Lenses for global brands. Now he runs a Coimbatore-based XR Studio, which helps brands use social XR for marketing campaigns and engage with their audience.
Recently Kavin started a London-based firm to target brands and opportunities in the European AR market. He employs five full-time employees and three freelancers, up from a single employee since last year. His team has worked for HBO for House of the Dragon, Camera Go, PS1 LyCA production, Zara, Coldplay Music, Tesco, The Great British Bake-Off, and Old Spice.
Gadgets 360: Do creators in India get a chance to work with international brands? Does Snap facilitate partnerships between creators and brands in the country?
Joe Darko: We have a creator marketplace where AR creators can get in touch with brands. Sometimes we have brands coming to us and we tell them to work with some creators to build out experiences, because we trust our creators who are closest to us to build these robust AR features for brands and businesses. This marketplace is pretty much like a self-service system where brands can go and find them.
Sometimes we connect brands that have relationships with Snap to AR creators and developers for new projects. We are in the business of democratising AR and that really started with Lens Studio being an easy and free-to-use developer platform. And as they get more competent and more comfortable, creators are able to grow with AR and are able to work with companies and make a living.
For example, a brand that wants to build an experience for users in India because they want them to build a local, relevant experience for the Indian market, would want to find someone in India. Then, we would connect them to creators in India. If they really want to hyperlocalise it, they can definitely do that. Take for example the partnership with HBO for the TV series House of the Dragon. Our teams were involved in picking the creators who would build that experience. And it was just amazing to see the global creator ecosystem come together and build it for HBO.
Gadgets 360: What about monetisation for Snap as a company? How will the company use AR for commercial product and brand placement on Snapchat in the future?
Joe Darko: We’re looking to build a healthy ecosystem, to help developers, make sure we have the right platform, while ensuring advertisers get what they want to continue to drive revenue for them. The key message here is not just Snapchat making money off AR as the ecosystem, but everyone benefiting from AR.
Gadgets 360: What are the challenges of supporting AR features on older or low-end hardware? Is there a set of minimum requirements that a user’s phone must have to use these features?
Joe Darko: It’s important to us that Snapchat users have a great experience using AR, regardless of what device they’re using. So, we invest deeply in going the “last mile” for our global community. We test and optimize our machine learning models and software so that Lenses preserve battery life, work consistently across platforms and device types – regardless if they’re built by our team or an AR creator. These investments make it easy for AR creators and developers to focus on building the best Lenses possible, and for Snapchat users to have the best experience possible, no matter the device they use.
Gadgets 360: Snap hasn’t announced any new hardware since the Pixy flying camera was shown off last year. Is the company working on any AR-specific hardware? Will we see new AR spectacles soon?
Joe Darko: I’m actually very glad you brought this up. We did bring a pair of Next Gen Spectacles to developers and they were actually AR glasses, but it was a dev kit for developers to build with us. We believe in being open with our developer community – when you do that, you’re able to get feedback from the community and you’re able to build with them and build experiences they will love.
So, we gave the developers who applied through the Spectacles programme a chance to test these AR glasses and build their experiences with them. It wasn’t available to consumers, just developers that we selected to build with us through the Spectacles programme.
As Snap we believe that users currently experience AR through their phones, but we also understand that the future of AR is in wearables and AR glasses. So, we’re definitely trying to iterate in that area and actually build glasses that we believe that developers and consumers will love.
I can’t tell you when we’re going to have an announcement for the launch of new AR glasses, but Snap doesn’t want to just ensure that AR works well on phones but also on head-mounted displays in the future.
Gadgets 360: We saw prototypes of new AR glasses from companies like Xiaomi and Oppo at the recently concluded Mobile World Congress (MWC) 2023. Does Snap plan to work with other hardware makers to support Snap AR features on their devices in the future?
Joe Darko: I’ll be honest and say we are open to the idea. We’re definitely open to build alongside the ecosystem, with ecosystem enablers. But I believe that we’re probably going to take on the same approach that we set first for the Next Gen Spectacles, which are our AR glasses for developers.
How do we build something for our community and our ecosystem first based on how we know it? If there’s room to work with different partners in the ecosystem, then I believe that from that perspective, we don’t want to leave any stone unturned, as a company. But at the same time, we’re figuring out how to do this ourselves. We want to learn from it and do better ourselves first.
Some responses have been condensed and slightly edited for clarity.