Workshop: Create your first Dataflow Flex template and set up a CI/CD pipeline for it on Cloud Build

Aug-5 18:00-20:00 UTC

Participation in this workshop requires additional registration and has limited capacity. If you are interested, please register here

This will be a 2h workshop with the following format:

  1. Introduction to Flex Templates, covering the following topics
  • What are Flex templates?
  • How do you create a Flex template?
  • How do you run a Flex template?
  1. Introduction to Apache Beam testing
  • Transform testing
  • End-to-end testing
  1. Developing and testing your first Flex template in Python

Given a DAG with the steps to include in a simple pipeline we will develop a Flex template to run it including all the required transform, unit tests for them and end-to-end tests for the pipeline.

  1. Setting up a CI/CD pipeline on Cloud Build

Once the Flex template is ready, tests included, we will create the build file for Cloud Build with all the required steps for our CI/CD pipeline and set up a trigger to a build every time the code is updated.