Tailoring pipelines at Spotify


Speaker(s):
Rickard Zwahlen

Tailoring pipelines at Spotify

By Rickard Zwahlen
Jul-18 10:50-11:15 in 204
Add to Calendar 07/18/2022 10:50 AM 07/18/2022 11:15 AM America/Los_Angeles AS24: Tailoring pipelines at Spotify

At Spotify we run more than 18,000 data pipelines. To help our developers build and maintain those, we made Scio - a Scala API for Beam, inspired by Apache Spark & Scalding. This session will show some of the advantages of Scio, and how we combine it with Beam for performance and ease of development.

 204

  Download slides

At Spotify we run more than 18,000 data pipelines. To help our developers build and maintain those, we made Scio - a Scala API for Beam, inspired by Apache Spark & Scalding. This session will show some of the advantages of Scio, and how we combine it with Beam for performance and ease of development.