From script slums to beam skyscrapers

Our data pipelines grew organically overtime one script piling up over another one. We got to a point where speed became too slow and too error prone to change. Sounds familiar? Come join me in this session where I would walk thro’ our journey to transform data processing from serial python scripts to massively parallel Beam.