Netflix's Tony Jebara on How to Use AI to Personalize the Customer Experience

Most Netflix users know anecdotally that there’s an AI-powered recommendation engine whirring in the background that drives what they see on their homepage and which shows are promoted to them. Your Amazon homepage and Google search results also feature this type of personalization now considered standard fare.

But many don’t know just how deep the level of personalization on Netflix goes, even down to the artwork you see for each show.

In fact, Netflix’s team of machine learning experts is working on an AI engine that can generate and provide personalized trailers and blurbs by analyzing the customer’s behavior on the platform to infer what they would be most likely to click and play.

Take the hit show, Stranger Things. Those who’ve seen it probably clicked on the title for different reasons – specifically, nine different reasons. At the O’Reilly AI Conference in New York City last week, Tony Jebara, head of machine learning at Netflix, showed the audience nine different thumbnails for the show which Netflix has shown to different users based on their inferred interests.

“Perhaps we realize you’re into teenage or younger shows, so it’s great for us to show you this image of two teenagers as the entry point into this show if you know nothing about it,” Jebara said, displaying a thumbnail of actors Natalia Dyer and Charlie Heaton, who play high school students Nancy and Jonathan.

Source: Netflix Tech Blog 

“Or if you like to watch scary horror movies, maybe we should show you this image of this slightly creepy-looking bleeding nose scene and that way intrigue you to think, oh, maybe there’s something there for me as well.”

The bleeding nose is in reference to the main character, Eleven, whose nose bleeds when she unleashes her telekinetic powers. While the user is browsing their Netflix carousel, a tip-of-the-iceberg view of a library containing over 14,000 movies and TV shows, each artwork has a split-second to capture the user’s attention and encourage them to click.

“When you look at your page what you’re seeing is an experience that’s been designed for you from top to bottom,” explained Jebara, an associate professor at Columbia University. “And the way it becomes unique is we use machine learning to figure out how to personalize the ranking of our entire catalog of movies and TV shows.”

Compelling artwork is even more important for new or lesser-known shows.

“Everyone knows Pulp Fiction, but if it’s a new show it’s very important to get the right image for the right person and get them to try it out,” Jebara added.

Once you’ve used the platform enough, Netflix starts to infer what else you might like. Jebara used two hypothetical user personas to demonstrate.

“Here’s a user upstairs who watches a lot of romantic shows and the best image for them from Good Will Hunting is this one of Matt Damon and Minnie Driver looking romantic together,” he said while displaying the artwork. “The user downstairs watches a lot of comedies. The best image for Good Will Hunting for them is the one which has Robin Williams because he’s a comedian.”

Netflix infers your preferences not just across genres; it’s more granular than that. If the AI detects an affinity for movies starring John Travolta or Uma Thurman, it will advance images containing those actors, Jebara added.

“If we use context, which is the member’s viewer history and the country they’re based in, we can learn better and find better images for that specific user.”

The world’s largest streaming network claims it doesn’t offer just one product but over 100 million products - one for each of its more than 100 million subscribers in over 190 countries.

Read more: Top Priorities for Investing in AI - Knowledge Management

In its early days, Netflix used A/B testing to determine the best artwork for each title, but found that its global customer base had tastes and preferences so diverse that it was impossible to universalize their preferences.

However, it did discover that “images that have expressive facial emotion that conveys the tone of the title do particularly well,” according to the official Netflix Tech Blog.

A major drawback of A/B testing is it requires you to expose a fraction of your customers to a potentially suboptimal experience while the experiment is ongoing, and it can take months to gather enough data for a statistically significant A/B test.

Intead, Netflix has designed algorithms that record and respond to customer behavior in real-time. Every time you click on a title, hover over a thumbnail, or watch a show, Netflix logs that behavior and measures how long you watch, what you add to your lists, and what you abort.

When technology serves a curatorial purpose that helps personalize the customer experience, like Spotify’s much vaunted playlist algorithms and North Face’s AI-assisted personal shopper, it’s a reminder that sometimes technology can provide a personalized and frictionless experience than the human touch.