How Airbnb & Yelp Test and Measure on Facebook for Optimal Advertising Performance — Recap from Smartly Connect SF

Laura Chamberlain Jan 04 2018 1 PM | 6 min read


Smartly Connect events are exclusive evening seminars that bring together the brightest minds in digital advertising. In late October, we invited some customers and partners to Hotel VIA in San Francisco’s South Beach neighborhood for talks, panels, and networking. For those that missed the event, don’t worry! We’ve compiled some interesting highlights and insights below.

A summary of the two panels: remember that 20% of your users is 80% of your revenue. When considering automation, do not automate experimentation: when looking into new strategies or creative formats, it is better to keep it manual to understand what is going on first. When considering multi-touch attribution, keep in mind that it's not only about figuring out what it would be cross-channels but also about providing insights about the data. With creative testing, be intuitive, fast, and intentional to get back high-quality data.


Always Be Testing & Measuring: Partners and Customers Discuss

Our Tracking & Measurement Partner Panel invited Ben Roodman of AppsFlyer and Katie Hutcherson Madding of Adjust to share the stage with our CEO, Kristo Ovaska. In addition, the Advertiser Panel put two of our most advanced customers in the hot seat to share their experiences on automating, optimizing, and testing campaigns. Moderated by’s own Trett Johnson, the audience had the chance to learn from Sam Ross of Airbnb and Ankur Damani of Yelp.

The 80/20 Rule and the Importance of Audiences

“The 80/20 rule is true,” stated Hutcherson Madding of Adjust. “20 percent of your users is 80 percent of your revenue.” It’s extremely important for advertisers to figure out how to leverage this and make use of the segmentation tools available to them, be it through mobile measurement partners or Facebook directly.

Damani of Yelp explained that when it comes to creative testing, they always start with audiences first. With such a broad product and many use cases, they’ve found success in focusing initially on audiences that have the most intent and working backwards from there, getting more granular as they go.

Both Appsflyer and Adjust warned not to be afraid of segmenting your audiences — testing smaller control groups can actually be a good thing and can significantly increase performance.

Ross explained that Airbnb also relies heavily on testing audiences, and experimentation. For that, they tend to stay away from automation tactics and interpret the data themselves to ensure they’re making the smartest decisions. 


To Automate or Not to Automate

Airbnb’s demand advertising consists mostly of Dynamic Ads. Ross explained that by treating these campaigns like product advertising, you won’t need to relaunch campaigns constantly (as they are automated). Supply is a different beast as they need to run tailored ads for every market.’s Automated Ads feature helps Airbnb scale and simplify the process; it creates prospecting ads automatically from any asset or destination feed.

Yelp also leverages Automated Ads in their day-to-day workflow. Their data feeds include images and content that gets reformatted into ads such as carousels that are then launched into local market campaigns. This way, they’ve managed to significantly reduce the amount of time it takes to pull together creative and help to test ads really quickly.

Damani stressed that creative really is key on social, especially when you only have a few seconds to make an impact in the eyes of a user. Automation helps them create and test new content constantly. The automated ad creation and launch is supported by’s creative rotation tool that ensures top performing and fresh creatives are looped in based on predetermined rules.

As a rule of thumb, you shouldn’t automate experimentation: when looking into new strategies or creative formats, it’s better to keep it manual to understand what is going on at first. The more data you have, the better the case for automation. Airbnb automates bidding & budget allocation for retargeting campaigns for instance, because those offer high quality data to base the decisions on.


Best Practices in Tracking and Measurement

Both Roodman and Hutcherson Madding have witnessed how multi-touch attribution has gained popularity among their clients in the past year. Roodman explained that the role of attribution is to offer the advertiser an unbiased view into the data, and to allow them to see the ROI and the effectiveness of their marketing spend. Many of AppsFlyer’s clients use mobile attribution in combination with cross-channel or omni-channel providers such as Adobe who offer integrated solutions. Roodman pointed out that with AppsFlyer, clients don’t necessarily need an app install objective in their Facebook campaigns in order to receive reporting on installs cross-channel.

Hutchinson Madding noted that all of their clients wanted multi-touch attribution in 2017. The increased need resulted in Adjust's ”Project Think Tank,” a brainstorming session where select clients, together with Adjust, discussed their needs more in-depth and brainstormed different ways to make those needs into something actionable. During the session, Adjust learned that their clients were primarily interested in:

  1. the percentage of networks that never get the last-click
  2. what percentage of clicks are attributed to top of the funnel
  3. how much of CRM initiatives are really getting through
  4. what percentage of traffic is actually unique amongst channels

Adjust understood that it was not attribution per se that clients wanted, but the different data points that it offers. In conclusion, it’s not only about offering Multi-Touch Attribution and figuring out what it would be on cross-channel, but also about providing insights about that exact data.


Creative Testing

We start with audience first,” says Damani from Yelp regarding creative testing. He explains that as Yelp is a broad product, it has many use cases, and thus there are audiences with more intent than others. With these audiences, Yelp focuses the profile down to age, gender, location and then works backward from there. The creative team takes that profile and tries to understand what is relevant to them.

There are, according to Damani, a few things that help make creative stand out in a feed. Firstly, they focus on original content. Damani says that Yelp has a lot of user generated content, which they try to use to their advantage. Second, they layer motion on top of static images and incorporate animation on top of video content. Damani has found motion to be an efficient way of communicating a succinct point quickly and converting intent into action. Lastly, Yelp focuses on the benefit and front-loading in their messaging, which garners a lot more engagement that they want to use to their advantage. 

Ross from Airbnb, on the other hand, emphasizes the importance of being intuitive and moving fast. He also explains how crucial it is to be intentional on what you are testing on creatives to get back high-quality data. By being very deliberate, Airbnb manages to keep costs and efforts low. Ross continues with sharing insights on how they sometimes do small tests at Airbnb; they throw a bunch of ads in an ad set and, for speed, let Facebook optimize them.

Creative Testing - watch the discussion with Airbnb and Yelp. 


To learn more about our solutions, visit our service page.




Laura Chamberlain

Read Next