Google Cloud Dataflow with Beam is at the core of how Credit Karma recommends the best offers for its 120M+ users worldwide. Using Scala with ZIO, we will demonstrate with code examples how to easily solve common production challenges including:
controlling misbehaving Tensorflow & PMML models
transient job failures and Dataflow bugs,
running multiple parallel jobs
detecting anomalous jobs
preventing errors at the petabyte scale
speculative jobs for optimizing operation
and more