Beam SDKs Don't Have to Look the Same


Speaker(s):
Robert Burke

Beam SDKs Don't Have to Look the Same

By Robert Burke
Sep-5 11:00-11:25 in Walker Canyon
Add to Calendar 09/05/2024 11:00 AM 09/05/2024 11:25 AM America/Los_Angeles AS24: Beam SDKs Don't Have to Look the Same

Do we even need PCollections? Or ProcessElements? Can we have the language fully typecheck the pipeline for us at compile time? Can we do that in Go?

Since Beam was designed, programming languages have continued to evolve and change, so why can’t our SDKs? We’ve now got ample experience with the Apache Beam Go SDK, but the language it was designed for is now very different.

This short talk will compare the current Go SDK with an experimental implementation that takes better advantage of the current strengths of Go, and its approach to generic type parameters, and more.

 Walker Canyon

  Download slides

Do we even need PCollections? Or ProcessElements? Can we have the language fully typecheck the pipeline for us at compile time? Can we do that in Go?

Since Beam was designed, programming languages have continued to evolve and change, so why can’t our SDKs? We’ve now got ample experience with the Apache Beam Go SDK, but the language it was designed for is now very different.

This short talk will compare the current Go SDK with an experimental implementation that takes better advantage of the current strengths of Go, and its approach to generic type parameters, and more.