PlusOne

In this talk we are going to cover how we have leveraged portability of Beam and make Stream Processing in Python possible on top of Apache Samza. We will first touch points on Apache Samza in general, and how it stands out as the stream processing engine at LinkedIn that scales to over a trillion messages processed per day, with strong state support and flexible deployment models. Next we introduce Samza Runner for Beam, particularly the portable runner. We will cover the efforts we have done to leverage Beam and make stream processing in Python available at LinkedIn, followed by a few use cases. The talk will conclude with our plan for future work.