Managed transforms - power of Beam without maintenance overheads


Speaker(s):
Chamikara Jayalath

Managed transforms - power of Beam without maintenance overheads

By Chamikara Jayalath
Jul-8 11:45-12:10 in Palisades
Add to Calendar 07/08/2025 11:45 AM 07/08/2025 12:10 PM BS25: Managed transforms - power of Beam without maintenance overheads

Apache Beam offers a number of powerful transforms including a set of highly scalable I/O connectors. Usually, you have to regularly keep upgrading Beam to get critical fixes related to such transforms. With the recent addition of Java and Python managed APIs, Beam allows runners to fully manage supported transforms. Google Cloud Dataflow uses this API to manage and automatically upgrade widely used Beam I/O connectors. This allows you to focus on the business logic of your batch and streaming pipelines without worrying about associated maintenance overheads.

Palisades

Apache Beam offers a number of powerful transforms including a set of highly scalable I/O connectors. Usually, you have to regularly keep upgrading Beam to get critical fixes related to such transforms. With the recent addition of Java and Python managed APIs, Beam allows runners to fully manage supported transforms. Google Cloud Dataflow uses this API to manage and automatically upgrade widely used Beam I/O connectors. This allows you to focus on the business logic of your batch and streaming pipelines without worrying about associated maintenance overheads.