Houston, we've got a problem: 6 principles for pipelines design taken from the Apollo missions

Jul-18 12:00-12:50 in 203
Add to Calendar 07/18/2022 12:00 PM 07/18/2022 12:50 PM America/Los_Angeles AS24: Houston, we've got a problem: 6 principles for pipelines design taken from the Apollo missions

Based in the inspiring talk about the Apollo XI “Light years ahead”, in this talk we will cover 6 design principles for data pipelines, with a focus on streaming pipelines. Submitting a job to a runner is not that different (well, a bit) from “submitting” a spacecraft into space. You lose direct access to the device, and can only send a couple of commands, read logs and receive telemetry. The 6 principles presented in this talk will make your pipeline to land on the moon rather than crashing!

203

Based in the inspiring talk about the Apollo XI “Light years ahead”, in this talk we will cover 6 design principles for data pipelines, with a focus on streaming pipelines. Submitting a job to a runner is not that different (well, a bit) from “submitting” a spacecraft into space. You lose direct access to the device, and can only send a couple of commands, read logs and receive telemetry. The 6 principles presented in this talk will make your pipeline to land on the moon rather than crashing!