PlusOne

Avro: Travel Across (r)evolution – Arek Osinski & Darek Eliasz

In those days, we are generating enormous amount of data. Biggest challenge is hidden in transformation of raw data to knowledge. We would like to take you on a short travel and show our approach for conversion from non-structured world of microservices to the world with Avro schemas inside our data pipelines.

Avro is well known format for storing and online processing information of any kind. What are key features of this format? What are the common problems? Where you can meet pitfails? How this influences our Big Data ecosystem?

Whole story will be covered by examples from real life implementation.

More information about this talk

Your Datascience Journey with Apache Zeppelin – Moon soo Lee, Anthony Corbacho & Jongyoul Lee

Take a journey together to see how Apache Zeppelin started, how Apache Zeppelin helps your data science lifecycle, how Apache Zeppelin became popular TLP project. We’ll also see how community focus has been changed, from basic notebook feature, spark integration to advanced features like multi-tenancy. Lee moon soo will explain value of Apache Zeppelin with some key use case scenario demo. Also we’ll see eco-system around it – How various projects and companies are using Apache Zeppelin in their product and services in many different ways.

Finally, we’ll discuss about Apache Zeppelin’s future roadmap with some challenges that community have.

More information about this talk

Scalable Private Information Retrieval: Introducing Apache Pirk (incubating) – Ellison Anne Williams

Querying information over TBs of data where no one can see what you query or the responses obtained? It sounds like science fiction, but it is actually the science of Private Information Retrieval (PIR). This talk will introduce Apache Pirk – a new incubating Apache project designed to provide a framework for scalable, distributed PIR. We will discuss the motivation for Apache Pirk, its distributed implementations in platforms such as Spark and Storm, itäó»s current algorithms, the power of homomorphic encryption, and take a look at the path forward.

More information about this talk

Building Streaming Applications with Apache Apex – Thomas Weise & Chinmay Kolhatkar

Stream processing applications built on Apache Apex run on Hadoop clusters and typically power analytics use cases where availability, flexible scaling, high throughput, low latency and correctness are essential. These applications consume data from a variety of sources, including streaming sources like Apache Kafka, Kinesis or JMS, file based sources or databases. Processing results often need to be stored in external systems (sinks) for downstream consumers (pub-sub messaging, real-time visualization, Hive and other SQL databases etc.). Apex has the Malhar library with a wide range of connectors and other operators that are readily available to build applications. We will cover key characteristics like partitioning and processing guarantees, generic building blocks for new operators (write-ahead-log, incremental state saving, windowing etc.) and APIs for application specification.

More information about this talk

Apache HBase: Overview and Use Cases – Apekshit Sharma

NoSQL databases are critical in building Big Data applications. Apache HBase, one of the most popular NoSQL databases, is used by Facebook, Apple, eBay and hundreds of other enterprises to store, analyze and profit from their petabyte-scale volume of data. This talk will discuss

– motivation behind NoSql databases

– basic architecture of a popular NoSql system, Apache HBase

– some commonly seen big data usage patterns in industry, and when & how to use Apache HBase (or other better suited NoSQL database).

More information about this talk