Speaker(s):

Dealing with order in streams using Apache Beam

Jun-14 10:30-10:55 in Palisades
Add to Calendar 06/14/2023 10:30 AM 06/14/2023 10:55 AM America/Los_Angeles AS24: Dealing with order in streams using Apache Beam

How to get data processed in order even when the queue deliver messages unordered?

One of the main problems when working with data in streaming is out of order. Data will come unordered and late. How come we apply a logic to that stream if it requires data to be ordered? Is there any solution to that?

In this talk, we will see how we can solve this problem using Apache Beam, and apply any temporal logic to keyed streams, however complex it is, even if it requires recovering the order in which data was produced.

Palisades

How to get data processed in order even when the queue deliver messages unordered?

One of the main problems when working with data in streaming is out of order. Data will come unordered and late. How come we apply a logic to that stream if it requires data to be ordered? Is there any solution to that?

In this talk, we will see how we can solve this problem using Apache Beam, and apply any temporal logic to keyed streams, however complex it is, even if it requires recovering the order in which data was produced.