The goal of this talk is showing a full example with a CI CD pipeline for Dataflow jobs.
The jobs will be based on Flex Template that is a way to standardize the deployment of Dataflow jobs and we are going to show a full example with Java and Python SDKs.
The CI CD will be orchestrated with Cloud Build based on a Github repository :
In an extra and optional part, we will show the Dataflow Flex Template deployment with Dagger IO and Go.
Dagger is a tool that allows to write CI CD Pipeline As Code.