Speaker(s):

New Avro serialization and deserialization in Beam SQL

Jul-18 17:15-18:05 in 204
Add to Calendar 07/18/2022 5:15 PM 07/18/2022 6:05 PM America/Los_Angeles AS24: New Avro serialization and deserialization in Beam SQL

At Palo Alto Networks we heavily rely on Avro, using it as the primary storage format and use Beam Row as in memory. We de/serialize billions Avro records per second. One day we realized Avro Row conversion routines consume much of CPU time. Then the story begins ….

204

At Palo Alto Networks we heavily rely on Avro, using it as the primary storage format and use Beam Row as in memory. We de/serialize billions Avro records per second. One day we realized Avro Row conversion routines consume much of CPU time. Then the story begins ….