Loading…
This event has ended. View the official site or create your own event → Check it out
This event has ended. Create your own
View analytic
Wednesday, October 28 • 3:40pm - 4:20pm
This is Sparkhara: OpenStack Log Processing in Real-time Using Spark on Sahara

Sign up or log in to save this to your schedule and see who's attending!

OpenStack service controllers produce large amounts of log data, and processing these logs can be a time consuming and difficult task. Data processing applications excel at automating these types of tasks and providing new windows into the underlying trends in the data produced.

We will discuss how to configure your stack to produce log data that can be consumed and analyzed in real-time by Spark applications running on the OpenStack Data processing service (sahara). These techniques can be used to inspect the current status and health of your services, decreasing the time spent between problem detection and solution. This also creates an exciting platform for developers to explore new transformations of log data that will unlock endless possibilities such as failure analysis, performance enhancement, and fraudulent activity detection.

This talk will cover a broad range of OpenStack and Apache technologies used in the development of this solution. We will discuss the following as they pertain to the greater problem space:


  • Logging service data to Zaqar and Manila

  • Launching Spark clusters with Sahara

  • Visualizing and experimenting on data with Zeppelin

  • Creating Spark jobs to deploy through Sahara

  • Storing processed data to Trove databases


There are many paths to achieving improved log processing in OpenStack. Attendees should expect to learn about how they can improve the stability and performance of their stacks through the advanced techniques discussed in this session.

Speakers
avatar for Nikita Konovalov

Nikita Konovalov

Software Engineer, Mirantis
Nikita has been working with OpenStack Data Processing (Sahara) from the early days of the project. He has been implementing the initial version of Sahara UI which then has been accepted to main OpenStack Dashboard codebase. He added support for Sahara benchmarking for Rally project and is now responsible for Sahara scale testing and benchmarking activities in Mirantis. In addition to it Nikita is contributing Hadoop HA support to the HDP plugin... Read More →
avatar for Michael McCune

Michael McCune

Senior Software Engineer, Red Hat
Michael is a software developer on Red Hat's Emerging Technology team, currently working on the Sahara project. He is also a member of the OpenStack API Working Group and OpenStack Security Team. He has been adding functionality and security enhancements to the Sahara project since the Icehouse release, as well as authoring the Data Processing chapter of the OpenStack Security Guide. He is currently working towards greater integration between... Read More →
avatar for Chad Roberts

Chad Roberts

Senior Software Engineer, Red Hat, Red Hat
Chad Roberts has been involved with Openstack Sahara since the Icehouse release.  In addition to work on Sahara, he has also contributed to the Horizon dashboard.


Wednesday October 28, 2015 3:40pm - 4:20pm
Expo Hall (Marketplace Theater)