Workshop: Create your first Dataflow Flex template and set up a CI/CD pipeline for it on Cloud Build


Speaker(s):
Miren Esnaola

Workshop: Create your first Dataflow Flex template and set up a CI/CD pipeline for it on Cloud Build

By Miren Esnaola
Aug-5 18:00-20:00
Add to Calendar 08/05/2021 6:00 PM 08/05/2021 8:00 PM America/Los_Angeles AS24: Workshop: Create your first Dataflow Flex template and set up a CI/CD pipeline for it on Cloud Build

Participation in this workshop requires additional registration and has limited capacity. If you are interested, please register here

This will be a 2h workshop with the following format:

  1. Introduction to Flex Templates, covering the following topics
  • What are Flex templates?
  • How do you create a Flex template?
  • How do you run a Flex template?
  1. Introduction to Apache Beam testing
  • Transform testing
  • End-to-end testing
  1. Developing and testing your first Flex template in Python

Given a DAG with the steps to include in a simple pipeline we will develop a Flex template to run it including all the required transform, unit tests for them and end-to-end tests for the pipeline.

  1. Setting up a CI/CD pipeline on Cloud Build

Once the Flex template is ready, tests included, we will create the build file for Cloud Build with all the required steps for our CI/CD pipeline and set up a trigger to a build every time the code is updated.

Participation in this workshop requires additional registration and has limited capacity. If you are interested, please register here

This will be a 2h workshop with the following format:

  1. Introduction to Flex Templates, covering the following topics
  • What are Flex templates?
  • How do you create a Flex template?
  • How do you run a Flex template?
  1. Introduction to Apache Beam testing
  • Transform testing
  • End-to-end testing
  1. Developing and testing your first Flex template in Python

Given a DAG with the steps to include in a simple pipeline we will develop a Flex template to run it including all the required transform, unit tests for them and end-to-end tests for the pipeline.

  1. Setting up a CI/CD pipeline on Cloud Build

Once the Flex template is ready, tests included, we will create the build file for Cloud Build with all the required steps for our CI/CD pipeline and set up a trigger to a build every time the code is updated.