Deep Dive on Apache Spark

There are a number of tools available for analyzing large data sets and over the last few years, one in particular has gained widespread popularity: Apache Spark. This Masterclass will present the concepts behind Spark and show you how to use it for analyzing your own data. It will use a publicly available dataset with some worked examples to demonstrate it in action.

In particular in this session, we will look at:

• What Spark is for and how to use it for data analysis
• How Spark processes data in a distributed fashion
• Using Spark SQL to query data interactively
• Using Spark’s Web GUI can be used to understand how it is running your queries on the data
• Building data pipelines for efficient processing of large datasets

This is a technical session and assumes you have:

• Knowledge of SQL/Databases
• Familiarity with programming languages (Python or other high-level language)

About the Presenter:

Stephen Oman is Director of Data Analytics in Travelport, a worldwide travel retail platform for travel agencies. He has been using Apache Spark for several years, building data pipelines for analysis of Travelport’s products

This masterclass is part of our ‘Technical Deep Dive Masterclass’ series delivered by our member organisations on a monthly basis. If you or your organisation would like to run your own masterclass for the benefit of our community, please get in touch.