Generative AI Meetup
By Beam Summit Team

Generative AI Meetup

In addition to the Beam Summit program, AICamp will be hosting the meetup: “Generative AI”. Generative AI is revolutionizing the developer, content creator landscape. Come to discuss and learn the generative AI as we dive into the world of LLMs, explore the capabilities of ChatGPT, and discover the power of text-to-image technologies. Immerse yourself in captivating discussions, demos, and practical applications led by industry experts. Join us https://www.aicamp.ai/event/eventdetails/W2023061414

Read More
Machine learning design patterns: between Beam and a hard place
By Beam Summit Team

Machine learning design patterns: between Beam and a hard place

In a recent book entitled Machine Learning Design Patterns, we captured best practices and solutions to recurring problems in machine learning. Many of these design patterns are best implemented using Beam. The obvious example is the Transform design pattern, which allows you to replicate arbitrary operations from the training graph in the serving graph while keeping both training and serving code efficient and maintainable. Indeed, the tf.transform package makes this easy.

Read More
Implementing Cloud Agnostic Machine Learning Workflows with Apache Beam on Kubernetes
By Beam Summit Team

Implementing Cloud Agnostic Machine Learning Workflows with Apache Beam on Kubernetes

The need for a highly efficient data processing workflow is fast becoming a necessity in every organization implementing and deploying Machine Learning models at scale. In most cases, ML teams leverage the managed service solutions already in place by the cloud infrastructure provider they choose. While this approach is good enough for most teams to get going, the long-term cost of keeping the platform running may be prohibitively higher over time.

Read More
Unified Streaming and Batch Pipelines at LinkedIn using Beam
By Beam Summit Team

Unified Streaming and Batch Pipelines at LinkedIn using Beam

Many use cases at LinkedIn require real-time processing and periodic backfilling of data. Running a single codebase for both needs is an emerging requirement. In this talk, we will share how we leverage Apache Beam to unify Samza stream and Spark batch processing. We will present the first unified production use case Standardization. By leveraging Beam on Spark for its backfilling, we reduced the backfilling time by 93% while only using 50% of resources.

Read More