TEAM LICENSES: Save money and learn new skills through a Hacking with Swift+ team license >>

How much is A/B testing driving app development at Netflix?

Recorded – watch the full episode on YouTube.

How much is A/B testing driving your plans to develop new features, versus relying on the gut instincts of team members?

Jordanna Kwok: Certainly both. I would say both because you might have a gut feeling about something, and to validate it you would do the A/B test. Often these A/B tests will fail, and we'll learn a lot from it. So this is another thing with the culture where making mistakes and failing is part of it and, yeah it might be my gut says something is going to be a great feature, and it turns out at the end of the day no one's using it.

"That's how we vet things out, and it's never like, "No, we don't want to do this", but it's more of, "Hey, let's try it and see what we learn from it."

So then you've learned something – maybe there's something that wasn't quite right, whether in the design or the engineering. And you could run it again; you can make tweaks and try it again and run the A/B test again, and when you get a positive signal, then great, we can roll this out. So, ideas come from everyone: engineers, designers, product managers, and more; it's not just the product managers.

Because typically people think of product managers as the keeper of the ideas, and they're the ones driving all these features. That's not necessarily the case. Lots of team members have pitched their own ideas and worked very closely with designers to flesh them out, and A/B test them, and a lot of times we're surprised when they come come back with a positive benefit for Netflix members using it. So that's how we vet things out, and it's never like, "No, we don't want to do this", but it's more of, "let's try it and see what we learn from it."

This transcript was recorded as part of Swiftly Speaking. You can watch the full original episode on YouTube, or subscribe to the audio version on Apple Podcasts.

Listen on Apple Podcasts

Hacking with Swift is sponsored by Superwall.

SPONSORED Superwall lets you build & test paywalls without shipping updates. Run experiments, offer sales, segment users, update locked features and more at the click of button. Best part? It's FREE for up to 250 conversions / mo and the Superwall team builds out 100% custom paywalls – free of charge.

Learn More

Sponsor Hacking with Swift and reach the world's largest Swift community!

Buy Pro Swift Buy Pro SwiftUI Buy Swift Design Patterns Buy Testing Swift Buy Hacking with iOS Buy Swift Coding Challenges Buy Swift on Sundays Volume One Buy Server-Side Swift Buy Advanced iOS Volume One Buy Advanced iOS Volume Two Buy Advanced iOS Volume Three Buy Hacking with watchOS Buy Hacking with tvOS Buy Hacking with macOS Buy Dive Into SpriteKit Buy Swift in Sixty Seconds Buy Objective-C for Swift Developers Buy Beyond Code

Was this page useful? Let us know!

Unknown user

You are not logged in

Log in or create account

Link copied to your pasteboard.