OpenStack Summit October 2015 Tokyo

As OpenStack itself has continued to evolve it's governance model, Neutron has done the same. In a world where we're all operating under "The Big Tent", Neutron has carved out it's own effort dubbed "The Neutron Stadium." During Liberty we've added a lot of new things into the project, from APIs to plugin and driver backends to new services.

This talk will cover a background on Neutron, how it's evolved as the OpenStack world has changed, what the Stadium has brought inside, and where things are headed. You will leave with a clear understanding of how Neutron is a platform, what has changed and what the future holds in this area.

OVN is a new network virtualization project that brings virtual networking to the Open vSwitch user community.  OVN includes logical switches and routers, security groups, and L2/L3/L4 ACLs, implemented on top of a tunnel-based overlay network.  For physical-logical network integration, OVN implements software gateways, as well as supports hardware gateways from a variety of vendors.

The OVN architecture simplifies the current OVS integration within Neutron by providing a virtual networking abstraction.  OVN provides Neutron with improved dataplane performance through shortcut, distributed logical L3 processing and in-kernel based security groups, without running special OpenStack agents on hypervisors.

In this presentation, we will provide an update on OVN's progress and provide a demo.  The demo will show an OpenStack-driven OVN deployment connecting containers and hypervisors.  We will demonstrate newly added features such as security groups and logical L3.  We will also discuss how users can start using OVN in their own deployments.

DVR (Distributed Virtual Router) was first implemented at Juno. Virtual routers needed to work on Network Node, but, with DVR, they can work on Compute Nodes.  Since Network Node runs virtual routers and some other networking functions for all VMs, it was a single point of failure and can be a performance bottleneck.  If you need a highly available OpenStack system, you needed a cluster of Network Node. And if you need to scale out the networking, you needed to add more Network Nodes.

With DVR, those issues can be addressed. At Juno, virtual routers were distributed to Compute Nodes, but DHCP and SNAT stayed put on Network Node due to some technical challenges. We still need Network Node for them, meaning we also need a cluster for HA and more Network Nodes for scaling out the system. Distributing them, we can complete DVR and we no longer need Network Node at all. We submitted a RFE for DHCP and SNAT distribution and worked on them, targeting for Liberty[1][2]. In this session, we will explain what were the technical challenges and how we solved them. Also explain how we can use the completed DVR and what the benefits are for us.

[1] https://bugs.launchpad.net/neutron/+bug/1468236
[2] https://bugs.launchpad.net/neutron/+bug/1467471

With the Liberty release of OpenStack, Neutron will have the ability to act as a BGP speaker on an operator's network.  This functionality enables Neutron to advertise host routes for floating IP's, and next-hops for tenant networks created with Neutron. This can alleviate many of the networking burdens placed on the operator when designing and operating an OpenStack cloud. It also enables new functionality in Neutron such as floating IP mobility and directly routable IPv4 and IPv6 tenant networks through a Neutron router.  In addition, deployment of Neutron BGP dynamic routing enables better support for existing features such as IPv6 and distributed virtual routing (DVR).

In this session we will explore how to use Neutron BGP dynamic routing to maximize the value of your Neutron deployment.  We will demonstrate how to deploy, configure, and use Neutron BGP dynamic routing and discuss the use cases it enables.  We will also explore how Neutron BGP dynamic routing works with features such as address scopes, routed network segments, distributed virtual routing (DVR), and what considerations operators should be aware of when deploying Neutron BGP dynamic routing.  We will also discuss future plans for expanding BGP dynamic routing capabilities in Neutron.

As IPv6 adoption continues to grow, there is an increasing need to deploy services in IPv6-only environments.  Openstack supports a myriad of configuration parameters for each service that can all be used together to create a deployment that requires only IPv6 addresses.

This talk will share experiences and lessons learned in deploying and operating an Internet-facing IPv6-only cloud, along with a demo showing its operation.  We will also give an outline on what needs to change, work that still needs to be done, possible future enhancements, as well as describe some new parameters in devstack that can be used to quickly get an IPv6-only test cloud up and running.

Service chaining allows dynamic steering of traffic from source, through a sequence of service function devices (physical or virtual), to destination. In current Data Center, the deployment of service chain for a tenant's flow is through static, complex, and rigid configurations and it usually takes days or weeks to deploy it. To reduce the OPEX and increase agility, automatic setup of service chaining is reuqired. 

Service Function Chaining (SFC) has long been a strongly requested feature in Neutron, and with the advent of Network Function Virtualization (NFV) as a major use case for OpenStack, that demand has only grown stronger. The wait for such feature is finally over in Liberty.

To address the needs, Service Chain feature is currently being developed as part of the Neutron Project. The community had gotten together and are developing a standard-based, vendor-neutral SFC functionality in Neutron using OVS as a reference driver implementation. In this talk we will give an introduction on the service function chaining project and its roadmap.

Attendees of this session will get to know:

1. Architecture Design, APIs, and Data Models of the Solution


2. Extensibility, Scalability, and Performance Consideration


3. Current Project Status: Where are we now?


4. Project Roadmap

In the Liberty cycle we worked on a new addition to Neutron, Quality of Service (QoS). In the network jargon QoS is about limiting, prioritizingor guaranteeing speed of traffic, in this case, on neutron ports.  The work in the Liberty cycle was focused mostly around the implementation of bandwidth limiting and laying out the QoS models to support future work.

In this session we'll cover the API changes and the new concepts that were introduced as part of this effort, like QoS rule, rule types and QoS policy.  We'll also share insights on how to get your Neutron driver to implement the API and make QoS available to your users.

Kuryr is a new project under Neutron's big tent that makes Neutron networking available to Docker containers by means of a Docker plugin.

In this session we will introduce Kuryr and show how it provides networking for containers in plain Docker environments and in mixed Docker, OpenStack environments.

Networking for container is rapidly evolving and overlay network is emerging as a popular choice given its simplicity.  One example is Flannel, a generic overlay network using IP datagrams on top of UDP.  Once deployed on a specific underlying infrastructure such as Neutron, there may be opportunities to optimize for performance and to ease some of the limitation of the overlay network.  In this talk, we look at the particular case of a Kubernetes cluster with Flannel running on Neutron networking and VM or baremetal.  This cluster would be provisioned by Magnum.

We will show how Flannel works with Neutron and Kubernetes' internal routing by tracing a round trip message.  We will quantify the overhead of Flannel and identify potential performance optimization.  We will look at the limitation of Flannel from the container perspective and show how some can be mitigated by Neutron.  We will show a demo for these features in a POC based on Magnum and Neutron.  Finally, we will discuss how the POC can be generalized to apply to other overlay  networks.

Of everything that we can build and deploy in a highly-available fashion in OpenStack, deploying highly available networking has been one of the trickiest, most complex aspects to get right.

In Neutron, when choosing the reference implementation, we have several components to consider when deploying for high availability:

• Neutron-server and its plugins are an API service as any other, with the same HA considerations, albeit with some aspects that deserve extra scrutiny when building for high availability


• Neutron L2 agents on compute nodes must ensure network connectivity for Nova guests


• and Neutron agents on network nodes must gracefully ensure routing and DHCP functionality even in the face of hardware failure

HA has also historically not been a very strong focus of upstream Neutron development, with vendors frequently filling the gaps with automation tools and smart service deployment.

In this presentation we cover:

• Upstream improvements on Neutron HA capabilities over the Juno, Kilo, and Liberty releases


• The approach taken for Neutron high availability in RHEL OSP and SUSE OpenStack Cloud


• Implementation details for Neutron HA in RHEL OSP and SUSE OpenStack Cloud

... All presented in the spirit of friendly competition, and upstream collaboration.

If you are an operator or user of DNS come learn about Designate, the DNSaaS provider for OpenStack. We will cover:

• Designate Architecture Overview


• Designate Use Cases


• Exciting features that landed in the Liberty release: ALIAS Records, Pool Scheduler, Horizon


• Integration with Nova and Neutron


• How Designate works any DNS server


• Customization points - API Extensions, Storage Plugins, Backend Plugin

We'll conclude with an exciting live demo showing how to add support for your favorite DNS server to Designate!

This talk is an update of the talk with the same name presented at the Openstack Design Summit in Hong Kong. Abstract and video for this talk are available respectively at http://openstacksummitnovember2013.sched.org/event/c6478ecf54d639de3b8b9958bfe9d450 and https://www.youtube.com/watch?v=RG6u_UuQOcs

Two years on, things have moved quite a bit.

Advanced services have advanced - and now they live in their own repos, plugin have been decomposed, and Neutron now is a stadium within Openstack's big tent.

This talk will go beyond choosing between a plugin and a ML2 driver, and will discuss the various options available to become part of the Neutron "stadium".

Monoliths versus modular plugins
Despite what one might think, this is not about ML2. Here we will discuss whether developers should develop Neutron integration a single monolitich plugin or as a set of plugins each one providing a different services.

Reasons for developing a ML2 driver
Apparently this is the only real option nowaday, but why? Are there cases where choosing to develop a ML2 mechanism driver is not really a no-brainer.

Extending neutron functionality through new services
The Neutron stadium offers different choices. We will look briefly looks at the services currently available in the "stadium", and provide useful insights into developing a new service, like the callback mechanism and ongoing efforts such as neutron-lib.

Where possible, examples targeting the "Human Defined Networking" Neutron plugin (https://github.com/salv-orlando/hdn) will be provided.

This talk has quite a good amount of technical content; the presenter is however committed to keep it lightweight, and, within the limits of the presenter's ability, entertaining.

This talk will discuss our experiences deploying Hierarchical Port Binding in production and at scale.  We'll compare an existing OVS+VXLAN implementation with a newer HPB+VXLAN based implementation.  We'll talk about the benefits of the newer implementation, as well as the problems we ran into along the way.

You just want to get to your instance by name.  But, Nova has your hostname, Designate has your DNS service, and Neutron has the IP allocation.  Could these ever be brought together to make it work like you want it?

In fact, they have come together.  In this talk, we'll talk about where we were at the end of Kilo and show you how we've come together to make this work across projects to give you the experience that you wanted.

"I can't ping my VM" , "Floating IP is not working" ....many people contact me with this kind of problems. That's why I decided to collect the most common hiccups of a Neutron deployment in this session,  provide some knowledge to understand the reason for the failure and a structured way to solve it.

Load balancing as a service has been one of the critical features asked for by cloud tenants.

For the Liberty release cloud providers such as Rackspace, HP, etc. have partnered with the community and load balancer vendors such as Radware and A10 to enhance the new LBaaS v2 service with Horizon support, L7 redirection, and a new service VM based reference implementation.


 

Neutron has a good feedback cycle between getting new features added and getting them tested by other contributors so they are functionally correct within one or two releases. Unfortunately, after a feature is working ("it worked in devstack!"), developers tend to move on to the next new thing. This happens well before the previous features are deployed at a large scale.

This delay has led to limited attention being paid to the performance of Neutron's HTTP management API and the AMQP control plane it uses to communicate with its agents. Subsequently, several cases had emerged over the past few cycles where operations were taking orders of magnitude longer than they should have to complete. These were imperceptible to developers because they were usually amplified by large numbers of networks and ports (instances) not present in a typical development environment.

This presentation will cover the wide range of performance improvements made to the Neutron management and control plane over the last ~6 months. These include both user-facing improvements (e.g. the time it takes to list Neutron networks) and deployer-facing improvements (e.g. the ratio of Neutron servers to L3 agents required to respond all agent requests). Benchmark and improvement numbers will be provided using measurements taken by Rally. This will be focused on the open source ML2 reference implementation; however, most of these improvements benefited many 3rd party plugins/drivers that utilize the same APIs.

Yahoo is huge! Yahoo has always had some legacy infrastructure in place, for better or worse, to solve the problems that OpenStack now solves. When we try to move to OpenStack, we can’t just get rid of existing infrastructure one fine day and tell people - “hey, I know that you have been using this or doing things this way for a long time.. From now on, you just can’t do that anymore. You need to update all your automation to work with this new thing”..

That will definitely piss off people.

Also, most of these legacy requirements come from our huge scale. So we need to make sure that when we move to OpenStack,  we continue supporting those use-cases. To do so, we need to patch Neutron heavily as it currently doesn’t support them. Most of them are useful for the community as well, since anyone at this scale is going to have similar requirement.

We are working with community to upstream them. With this talk, we would like to share our experience, use-cases, hacks and also like to know if anyone else in the community has the same requirements.

We need following features to support our legacy use-cases

• Static IP allocation: Currently, when inventory is installed in racks, our site operations team picks IP address for the inventory and updates our inventory management database. Currently, we consider this database as source-of-truth. When this inventory is added to Ironic and booted, Neutron first looks at its original IP address, creates networks/subnets on-the-fly, if not already present and then assigns the same IP to the instance. Currently we have to do this because we can’t yet treat Neutron as the source-of-truth for bare-metal inventory. We are slowly trying to move towards Dynamic IP allocation where Neutron will be the source of truth, but there are certain blockers that we need to solve first.


• Single process Dnsmasq support for Neutron DHCP Agent: At Yahoo we don’t have tenant networks or overlapping IP spaces. So we don’t need dhcp-agent to spawn a Dnsmasq process for every network. We have patched the agent and the driver to spawn a single process responsible for all the networks. We don’t plan on using this patch forever, but only temporarily until we get support for ISC-DHCPD driver in.


• Support for ISC-DHCPD driver for Neutron: Dnsmasq does not scale well, especially for a scale at which Yahoo operates. Also, ISC-DHCP is mostly what is used at this scale. Currently Neutron DHCP agent doesn’t support any other DHCP server that Dnsmasq. In fact, the agent itself is so aligned with Dnsmasq, that it doesn’t work as-is for ISC-DHCP. So we have modified the agent and added a driver for ISC-DHCPD. This work is being upstreamed. 


• Support for multiple gateways for a subnet in Neutron: At Yahoo, the network architecture is such a way that multiple gateways are configured for the subnets. These multiple gateways are typically spread across backplanes so that the production traffic can be load-balanced between backplanes. Currently, the subnets in Neutron only support one gateway. We are working with community to add multiple gateway support to Neutron subnet.


• Multi-IP support: For one of our use-cases, we need ability to allocate multiple IPs for an Ironic instance, but not necessarily create a port for it in Neutron. Currently we are evaluating different ways we can do this.


• IPv6 support on Opt-In basis: We need Neutron to be able to allocate IPv6 addresses, but not always. We have some legacy stuff that will simply not work with IPv6. So we need provide ability for users to “Opt-In” for either only IPv6, only IPv4, or both. This is also something that community doesn’t support as of now. We would like to know if others also have similar use-cases, so that we can work with them to upstream this.

If you are an operator or user of CDN, then come learn about Poppy - the CDNaaS API for OpenStack.

We will cover:

• Poppy Architecture Overview


• Poppy Use Cases


• CDN Vendor Drivers that are part of Poppy


• Exciting new features that have landed: Multiple Domains/Origins, TTLs, Host Headers, SSL, Referrer/Geo/IP Restrictions


• How Poppy works with your preferred CDN vendor


• Customization - Vendor Plugins, Storage Backends, DNS Plugins