Beam YAML: Advanced topics

While the basic features of Beam YAML allow one to write simple to possibly intermediate pipelines, there are limitations on what can be developed relying solely on the built-in transforms and basic features. Luckily, Beam YAML had these users in mind during its development and offers multiple ways to leverage more advanced features of Beam to implement these sophisticated use-cases.

The purpose of this session will be to dive into the more advanced features Beam YAML has to offer. Topics include, but are not limited to:

• Defining explicit transform output types
• Advanced mapping
• Advanced aggregation
• Transform Providers
• Inline Python
• Jinja preprocessing
• ML transforms

If time allows, there will be some use-cases demonstrating the features presented above.

This session builds upon the information presented in the introductory session and it is recommended that attendees view that session before diving into the topics presented in this session.