Scaling Public Internet Data Collection With Apache Beam

In Cortex Xpanse, we are scanning the internet and collecting public internet data in order to identify issues in customers’ assets.

In this session, I’ll present the wide usage of Beam in our org (hint: we have over 1000 daily running jobs) and share our best practices. Be ready to learn how are we deploying, monitoring, optimizing, and testing our pipelines.