OpenStack Summit October 2015 Tokyo

CloudFoundry (CF) is a Platform-as-a-Service (PaaS) that is designed to be agnostic to infrastructure-as-a-service (IaaS) clouds and application platforms. This means that CF can be deployed in many IaaS, e.g., AWS, OpenStack, SoftLayer, and deploy different applications of varied platforms, e.g., Ruby-on-Rails, Python-Django, Golang, PHP/Zen, and so on.

CF achieves it’s cross-cloud capabilities by defining a thin layer called Cloud Provider Interface (CPI) that is implemented for each targeted cloud and is used by a CF cloud tooling called BOSH to help create, manage, and maintain IaaS resources such as VMs, IPs, and storages.

One of the key targeted clouds in the CF community are those supporting OpenStack. Using an OpenStack CPI we should in theory easily deploy CF, which consists of at least 10s of VMs with sometimes dozens of running jobs (long running processes), onto OpenStack clouds.

While the results are achievable in theory, in practice, what we have seen is that the actual deployments and maintenance of CF installations in different OpenStack providers of the same version result in changes and differences that bleed into the CPI layer. This means that the same CPI needs to be changed even though we target two OpenStack clouds supporting the same API version!

In this talk we describe the CF BOSH CPI layer and the various reasons why in our experiences we are still not able to perfect cross-cloud deployments of large systems, such as CF, in different OpenStack providers and versions.

In this talk we'll give you an overview of a platform, called Software Factory, that we develop and use at Red Hat. It is an open source platform that is inspired by the OpenStack's development's workflow and embeds, among other tools, Gerrit, Zuul, and Jenkins. The platform can be easily installed on an OpenStack cloud thanks to Heat and can rely on OpenStack to perform CI/CD of your applications.

In this session, you will learn how to:

- Deploy a CI/CD platform (Software Factory) on OpenStack
- Manage your CI/CD workflow thanks to Zuul
- Manage your slave nodes
- How to export CI/CD jobs's logs on Swift

OpenVSwitch recently added Open Virtual Networking (OVN) to OVS. We'll cover that and other recent changes to OpenVSwitch. Emphasis on Kilo and Liberty and changes in OVS and OVN.

The storlet framework enables running user-defined functions, such as transformations and filtering of data as it is uploaded or downloaded to/from an object store.  We have integrated the storlet framework with Swift using the standard 'middleware way'. Unlike conventional Swift middleware though, storlets provide a framework to run a dynamically loaded computation on the Swift data path, where the computation executes either on the object or the proxy nodes inside a Docker container.

Calling for participation, we are releasing an initial reference implementation of the storlet framework, which uses Docker containers, as a Stackforge project.  In addition, the related Swift middleware has been submitted to the Swift community for a review.

In this talk we will review the project, detail the design of the storlet framework, show use cases for analytics and media, and outline the future plans for Storlets, including potential integration with container management frameworks like Magnum as well as other integration options.

This talk will cover the pros and cons of four different OpenStack deployment mechanisms. Puppet, Chef, Ansible, and Salt for OpenStack all claim to make it much easier to configure and maintain hundreds of OpenStack deployment resources. With the advent of large-scale, highly available OpenStack deployments spread across multiple global regions, the choice of which deployment methodology to use has become more and more relevant. 

Beyond the initial day-one deployment, when it comes to the day-two and beyond questions of updating and upgrading existing OpenStack deployments, it becomes all the more important choose the right tool. 

Come join the Bluebox and IBM team to discuss the pros and cons of these approaches. We look at each of these four tools in depth, explore their design and function, and determine which scores higher than others to address your particular deployment needs. 

Since open sourcing MidoNet at the OpenStack Paris summit, many production deployments of OpenStack have selected MidoNet for their networking.  Over 20 companies are now involved with the open source project, and it's quickly growing in popularity.

In this session, I'll cover an introduction to MidoNet, an open source network virtualization overlay (NVO) plugin for Neutron. We'll do a full overview of the architecture, features, and give a demo showing off some of the advanced functionality MidoNet can offer, like distributed load balancing, and firewalls.  You'll also get an overview on how to get started deploying MidoNet in as little as 20 minutes.

By the time you leave this session, you'll have a jump start education on one of the coolest Neutron plugins around.

Visualizing OpenStack data is one of the most powerful ways to make sense of what's going on with cloud-based resources. The OpenStack Horizon dashboard provides a number of built in visualizations, but how do we move beyond the features that are provided by Horizon to develop deeper or richer visualizations?

This presentation is based on our experience of developing the open source project Goldstone, and will discuss some of the challenges of making sense of OpenStack data, as well as demo some of the capabilities of D3.js.

One of the solutions our team has come up with for moving beyond Horizon was to develop software with an api-driven architecture that uncouples the data visualizations and the underlying data. Our client is built upon Backbone, a popular JavaScript framework, that sends ajax requests to the server component which is installed alongside an OpenStack cloud. One of our main challenges has been determining what data to gather, and how to make it relevant to an OpenStack operator. In addition to gathering metrics, we’ve developed a resource graph with the goal of finding the upstream source of problems in your cloud.

The software client we’re developing is built entirely in JavaScript, taking advantage of the Node.js JavaScript developer ecosystem tools such as Grunt, Karma, and Sass. And for the visualizations, we use D3.js, a low-level, highly customizable library for creating flexible, data-driven visualizations.

This presentation will include a walk-through of our learnings from developing Goldstone, a summary of some of the challenges of visualizing OpenStack our team has overcome, as well as a demo of some of the capabilities of D3.js.

Zaqar is the messaging and notifications service for OpenStack that allow users to build scalable, reliable and high-performing applications. It has a REST API that can be used by developers to transport messages between different components in their cloud applications through different communication patterns, such as producer/consumer or publisher/subscriber.

In Zaqar's v1, only a WSGI controller is available as a transport solution. This controller is enough for most of the use cases required by users like, for example, task distribution, broadcasting and point to point messaging.

But when the storage and recovery of messages is incremented to rates over 10.000 messages per second -- in cases like activity tracking in a website or log aggregation -- this transport is very inefficient.

To accomplish an adequate performance for this type of loads is necessary the implementation of a streaming interface. That is, long living connections with minimum overhead in which the messages can be freely pushed.

The websocket driver, the latest addition to the Zaqar project, aims to cover this use cases.

Attendees will gain insight into Zaqar's architecture before and after the API refactoring and the new driver addition, how we implemented the driver,  information about Websocket and why we choose it as a transport solution, along with some uses cases and benchmarks.

Getting your cloud images right is critical to the success of any cloud deployment. Getting all of the basics in place from security settings to patch levels will greatly streamline your use cases down the line. Doing this by hand can be complicated and filled with potential missteps.  The key to mitigating this is automating the process with a tool that is both reproducible and testable.

Enter Packer!

Packer is a tool developed by HashiCorp (the people behind Vagrant) to help you create identical cloud images for a variety of different environments. It also allows you to create image templates that are easy to version control and understand what happens during the image creation process.

In this talk, we will show you how to get up and running with Packer and get your images right from the start. We will cover the following areas:

• Packer Installation -  We will show you how to get Packer up and running quickly


• Preparing OpenStack - We will review what is needed on your OpenStack cloud to get started.  


• Building your first Packer Template - We will create and review the options in a Packer Template


• Testing - No code is complete without it.


• Expanding your Template with provisioners - We will review the options for iteration, once your base template is in place.

Packer has many options and will work with OpenStack and many other cloud solutions, both public and private. Join us to see how this tool really takes a lot of the pain out of image building images.

Monitoring and log management are two closely related siblings, necessary to guarantee the stability and availability of an OpenStack system. Monasca is a scalable, multi-tenant, high performance, fault-tolerant OpenStack Monitoring as a Service (MONaaS) solution described at https://wiki.openstack.org/wiki/Monasca. In this session we present our work on integrating additional log management functionality into Monasca, to bring the two siblings together in one tool - tailored for OpenStack'ers.

The technical basis of our work consists of Elasticsearch, Logstash and Kibana, aka ELK. The ELK stack has probably become the most widely used open source tool for centralized log management. It also finds frequent adoption among OpenStack users, as demonstrated in various sessions at previous summits. The goal of our work in the Monasca project is to:

1. give operators a unified turn-key solution to manage all logs from their multiple OpenStack systems;
2. provide OpenStack tenants a service that allows for managing logs from their applications running on OpenStack - accessible from Horizon.

As part of this talk we will dive into our architecture to show how to meet our primary design goals: scalability, multi-tenancy, resilience and extensibility. One important aspect is our integration with Monasca that uses common components, such as Kafka, and common architectural patterns. Another highlight that we will point out is our newly implemented REST API, that facilitates Logging as a Service in the spirit of Loggly. Here, authentication happens via Keystone, which extends the existing ELK stack by adding multi-tenancy, applying the tenant model of OpenStack.

Finally, we will give a demo of our solution.

Keywords: monasca, elk stack, elasticsearch, fujitsu, hp, logging, log management, monitoring

Ceph is a fully open source distributed object store, network block device, and file system designed for reliability, performance, and scalability from terabytes to exabytes. Ceph utilizes a novel placement algorithm (CRUSH), active storage nodes, and peer-to-peer gossip protocols to avoid the scalability and reliability problems associated with centralized controllers and lookup tables.

The community has been very active integrating Ceph into OpenStack. This is getting better each release.

With Juno we reached a critical step in terms of feature, robustness and stability. Kilo is even better, and Liberty is promising but we are not done yet.

In this session, Sebastien Han and Josh Durgin from Red Hat will describe the current state of the integration of Ceph into OpenStack and where are we heading to in terms of roadmap. They will go through all the OpenStack projects.

One of the most recent addition to Sahara (OpenStack Data-Processing-as-a-Service component) was the Storm plugin. Up to Juno, Openstack Sahara included only batch processing alternatives and Storm is one of the most popular open-source tools for real-time data analytics and stream processing.

Implementing real-time data processing differs from Hadoop and other batch processing approaches because the data cannot be stored and then processed. The processing takes place while the data traverses the system. As a consequence sub-second processing latencies can be achieved. Therefore, this plugin enables new types of applications to be executed in Sahara.

Real-time data processing is increasingly popular. Many such applications are now in our daily routine. One example, is online data summarization, where a high volume data feed (e.g., logs from a large cluster or sensor network) is summarized to let only relevant events be stored in a database. Other common examples, with varying performance and scalability requirements, include fraud detection, trend topics, and high frequency trading.

In this talk, we will present the Storm plugin for Sahara and guide the user through the essential steps to setup a scalable real-time data processing application. We will also share our plans for improving real-time data processing in OpenStack.

The work of running OpenStack has been getting easier; unfortunately, it's still just as hard to operate the underlying physical data center.  In this presentation, we're going to talk about how to use open source tools build a consistent and repeatable underlay for your OpenStack infrastructure using OpenCrowbar and OpenContrails.

Most importantly, we'll talk about how to TEAR IT ALL DOWN AND START OVER.

The RackN and Juniper teams have been collaborating to create an automated ready state environment that is hardware,  distribution and devops tooling agnostic with both physical and network topologies automatically configured for OpenStack environments using open source technologies.

In this session, we'll start with some first principles on physical Ops to ensure OpenStack success then we'll show how to duplicate this experience with your own infrastructure.