Data-Science

TL;DR Getting a response from GenAI is quick and straightforward. But what about the... The post Building Confidence: A Case Study in How to Create Confidence Scores for GenAI Applications appeared first on Spotify Engineering.

On Spotify’s Analytics Platform, we’re dedicated to building products that empower data practitioners to discover, analyze, and share insights — [...] The post Are You a Dalia? How We Created Data Science Personas for Spotify’s Analytics Platform appeared first on Spotify Engineering.

We have a lot of dashboards at Spotify. Our Insight teams and analysts from across the company are constantly whipping [...] The post Unlocking Insights with High-Quality Dashboards at Scale appeared first on Spotify Engineering.

Check out Data Platform Explained Part I, where we started sharing the journey of building a data platform, its building [...] The post Data Platform Explained Part II  appeared first on Spotify Engineering.

TL;DR Sometimes we cannot estimate the required sample size needed to power an experiment before starting it. To alleviate this [...] The post Fixed-Power Designs: It’s Not IF You Peek, It’s WHAT You Peek at appeared first on Spotify Engineering.

As engineers working at Spotify, we frequently find ourselves explaining our robust data platform to fellow professionals who are contemplating [...] The post Data Platform Explained appeared first on Spotify Engineering.

TL;DR We summarize the findings in our recent paper, Schultzberg, Ankargren, and Frånberg (2024), where we explain how Spotify’s decision-making [...] The post Risk-Aware Product Decisions in A/B Tests with Multiple Metrics appeared first on Spotify Engineering.

Large sets of diverse data present several challenges for clustering, but through a novel approach that combines dimensionality reduction, recursion, and supervised machine learning, we’ve been able to obtain strong results. The post Recursive Embedding and Clustering appeared first on Spotify Engineering.

For the past decade, Spotify has used approximate nearest-neighbor search technology to power our personalization, recommendation, and search systems. The post Introducing Voyager: Spotify’s New Nearest-Neighbor Search Library appeared first on Spotify Engineering.

What is “Exclude from your taste profile”? Are you a parent forced to put the Bluey theme song on repeat? Do you work from home and play lofi beats or ambient piano music? Do you fall asleep to peaceful ambient noises? Are you bummed out when these songs come up as your most listened to [...] The post Exclude from Your Taste Profile appeared first on Spotify Engineering.

When we want to determine the causal effect of a product or business change at Spotify, A/B testing is the gold standard. However, in some cases, it’s not possible to run A/B tests. For example, when the intervention is an exogenous shock we can’t control, such as the COVID pandemic. Or when using experimental control [...] The post How to Accurately Test Significance with Difference in Difference Models appeared first on Spotify Engineering.

At Spotify, we run a lot of A/B tests. Most of these tests follow a standard design, where we assign users randomly to control and treatment groups, and then observe the difference in outcomes between these two groups. Usually, the control group, also known as the “holdout” group, retains the current experience, while the treatment [...] The post Encouragement Designs and Instrumental Variables for A/B Testing appeared first on Spotify Engineering.

As companies mature, it’s easy to believe that the core experience and most user needs have been resolved, and all that’s left to work toward are the marginal benefits, the cherries on top. Cherries on top might add delight and panache, but they rarely cause fundamental shifts in performance and success. And as a business, [...] The post Experimentation at Spotify: Three Lessons for Maximizing Impact in Innovation appeared first on Spotify Engineering.

TL;DR: Spotify is releasing a new commercial product for software development teams: a version of our homegrown experimentation platform that we’re calling Confidence. Based on everything we’ve learned over the last 10+ years about what it takes to enable experimentation at scale, the platform makes it easy for teams to set up, run, coordinate, and [...] The post Coming Soon: Confidence — An Experimentation Platform from Spotify appeared first on Spotify Engineering.

In Part 1 of this series, we introduced the within-unit peeking problem that we call the “peeking problem 2.0”. We showed that moving from single to multiple observations per unit in analyses of experiments introduces new challenges and pitfalls with regards to sequential testing. We discussed the importance of being clear about the distinctions between [...] The post Bringing Sequential Testing to Experiments with Longitudinal Data (Part 2): Sequential Testing appeared first on Spotify Engineering.

Spotify’s approach to challenges in sequential testing with longitudinal data  At Spotify, we’re constantly improving our data infrastructure, which means we can get feedback on experiments earlier and earlier. To allow for early feedback in a risk-managed manner, we use sequential tests to monitor regressions in the experiments. However, when moving toward smaller and smaller [...] The post Bringing Sequential Testing to Experiments with Longitudinal Data (Part 1): The Peeking Problem 2.0 appeared first on Spotify Engineering.

Messaging at Spotify At Spotify, we use messaging to communicate with our listeners all over the world. Our Messaging team powers and creates delightful foreground and background communications across the Spotify experience, experimenting with and tailoring the perfect journey for each user across our platform. Today we are able to send messages through WhatsApp, SMS, [...] The post Experimenting with Machine Learning to Target In-App Messaging appeared first on Spotify Engineering.

Introduction In the fast-paced world of streaming, personalization plays a vital role in enhancing user experiences. At Spotify, our Home serves as a personalized surface where users retrieve familiar content and discover new content tailored to their preferences. We are constantly trying to optimize the Home experience to provide more value to our users; however, [...] The post Experimenting at Scale, the Spotify Home Way appeared first on Spotify Engineering.

Navigating tradeoffs between different products

Questions to ask and other tips