User login

Richard Sanger's blog




I've done some basic benchmarks (for loops around operations) and made some optimisations based upon that for the libraries I've been testing. All the libraries performed well enough for my purposes. The performance of the libraries did differ depending upon the task being performed, with some outperforming the others at one task and the reverse with another task. Overall I've found ROFL to be the easiest to work with and it appears to still be in active development.

I've have now got access to the OFLOPS-Turbo source. So I will be looking at that to see what changes have been made and if it is more suitable to build on top of.

I've focused my attention back to RouteFlow again and started to fix some problems. I've investigated the issue with discovering hosts (i.e. ARP). Previously we were not sending IP traffic to the controller meaning that we did not already know about a host we would drop all traffic destined to it. I investigated using broadcast (ping, arp, dst mac broadcast etc.) methods to try find all hosts proactively however this did not work as most machines were not listening. As such I concluded we still have to send unknown IP traffic to the controller so it can perform ARP. This is a DOS vector, but should be less of a concern now with fastpath.




I've continued evaluating libraries.

I run into issues with padding using loxi, turns out this needs to be done manually and in the correct order to ensure fields are not overwritten etc.

I looked at using OFConnect again this seems to perform similar functionality to fluidbase, helps establish a OF connection. However does not include functions to build OF messages, given I'm happy with fluidbase and that there are a few bugs listed with OFConnect I've decided to skip it in favour of Revised OpenFlow Library (ROFL).

ROFL is a C++ library and supports fewer OF versions than loxi, however supports 1.0 and 1.3. The same flow mod is used for both allowing for quite nice support between versions.

I've also had a quick play with floodlight's oftest. A python library that tests OpenFlow switches capabilities and conformance to the OF standards. It's similar to OFLOPS however appears to be targeting testing capabilities rather than benchmarking performance. I.e. the tests included typically add a rule and send a packet that will match it and verify the packet is returned modified correctly. I needed to make some slight modifications to get it working such as removing, including adding some sleep to work around what I'm assuming is incorrect barrier behaviour (response before TCAM update) and some removed some specific match fields as it creates very specific matches matching every field in the packet to be sent.

Brad got the Brocade working, I've done some quick testing it appears to work but has limited support for features and lacks VLAN support. However it would still be useful to test packet in rates upon.




This week I've been continuing to look at OFLOPS (a testing framework for OpenFlow switches) and enquiring about hardware to test upon. Josh Bailey has put me in contact with the original OFLOPS authors, so I will work with them to figure out the best way to update OFLOPS.

I've been looking into what the best library is for constructing/parsing OpenFlow messages on top of a measurement platform such as OFLOPs. I'm looking for something in a low level language such as C/C++ for performance reasons. However I also need to support at least OpenFlow 1.0 and OpenFlow 1.3. Hopefully the differences between the two will be abstracted as much as possible allowing the same code to create both OF 1.0 and 1.3 messages (and any future versions). Ideally the code is going to be concise etc.

I'm currently looking at libfluid, (floodlight's) loxigen and OFConnect. I'm currently writing some simple cases in each to see how easy they are to use.




I've continued researching into SDN in general. I've been going through a large survey "Software-Defined Networking: A Comprehensive Survey" which has a very good list of SDN projects and how they fit together (It references 500+ papers).

I've continued to look into research regarding PACKET_INs as far as I can tell there is practically no published work on the performance of PACKET_INs.

I've been considering how best to put together a testing platform. Tools such as OFLOPS exist and allow modules to be written to test various elements of OpenFlow switches. Interestingly OFLOPS already has a PACKET_IN module i.e. what I'm wanting to test. However OFLOPS no longer seems to be maintained and only supports OpenFlow1.0. I also run into other issues/crashes when I tried to run it.

My main options at the moment are to start from scratch/update OFLOPS or hack together a one off system for this testing. I would prefer the former as this could let me test other aspects and in the long run would probably end up faster.




This week Brad sliced of a portion of the pronto for me to experiment with, an OVS bridge with some ports attached. The processing resources on the pronto are still shared with the production RouteFlow and Valve instances.

I encountered a few minor issues with rule priorities and VLANs and some bugs with my addition to RouteFlow.

From preliminary testing it seems that the PACKET-IN performance of the pronto is very low. This seems like it is related to exhausting the CPU time on the switch.

I also started looking to see if I could find any other research into this, so far research seems primarily to be targeting the performance of modifying OpenFlow rules.




I've primarily been spending my time working fastpath into RouteFlow. This has not been as simple as I expected because RouteFlow itself does not control the rules on the OVS between the lxc that's doing the routing and the controller machine ('dp0'). After a discussion with Brad I choose to leave this separation. And I have written a tool which parses the configuration and sets up dp0 correctly. I've modified the Vandervecken setup scripts to include the necessary rules, which are loaded from the RouteFlow configuration files.

I've read over a paper recommended by Richard Nelson, "What You Need to Know About SDN Flow Tables" which has some interesting points about the performance and correctness of switches available currently. Both of which have some substantial room for improvement.




I spent last week researching and reading papers in the field of SDN. I've been writing summaries of the papers I'm reading.

There seems to be some research around how to best write SDN applications which interests me. In particular there is a notion that OpenFlow is to SDN what assembly is to software programming.

Even with my brief experience so far with my in progress fastpath addition to RouteFlow I felt it would be nice to have someway to add this as a modular addition - without extensive modifications to RouteFlow. Particularly given that essentially any OpenFlow application could benefit from fastpath, it would be nice not to re-write this into every application.

Their are some solutions to help solve this currently one such is frenetic, a language, which allows for serial and parallel combinations of network rules. This could be used to link functions together first apply firewall then route etc. Other abstractions such as network topology virtualisation allowing multiple physical switches to be treated as one single OpenFlow switch. This allows for much higher level control, focusing on only what the network should do.

CoVisor achieves something similar to frenetic, however is a hypervisor rather than a language. CoVisor provides an OpenFlow interface allow multiple controllers to connect and be combined together in a similar manner to frenetic, in parallel, serial or overridden. The idea being you deploy the best of any controller application on a network no matter what language it is written in.




Updated fastpath to use tables, this now requires only an extra rule per port rather than an extra rule per port for every send to controller rule. Moved to using the dpset events in ryu, rather than the direct OpenFlow events, allowing access to more information in a single event. The next step is to move this into RouteFlow. Chris Lorier mentioned this work to Josh Bailey, who is interested in it and has suggested an approach to integrating this into RouteFlow. The work so far is now on Github -

I've started reading through and watching talks on Googles WAN SDN development, B4, and the reasons for it used to connect their data-centres around the world. Among other factors some of the main ones where to keep costs down by allowing them to run links close to 100% utilisation as opposed to the typical 30-40% WAN's are traditionally provisioned at. Supporting for better Traffic Engineering has allowed for better handling of failure scenarios and achieve overall better balance of traffic across the network compare with shortest path first routing.




This week I focused on the fastpath problem and have created a separate ryu controller and connected both the internal RouteFlow OpenFlow switch, dp0, and my real mininet switch to it. This has been a good way to learn the OpenFlow specification through practical use, the most popular protocol use to control SDNs. I've also had to learn ryu an OpenFlow python library for creating OF controllers and skill up on my python.

During this I've encountered some minor issues
* Originally to connect the controller and network with a fastpath in my emulatation environment I tried to use a Linux tun device however found that I could not add this to ovs. However Linux tap devices work - I suspect the lack of a MAC address on the tun was the cause of this.
* I've have run into problems trying to using MPLS labels to hold port information, I'm unsure if this is an OVS issue, something I'm doing wrong or an issue with trying to put these through a Linux tap tunnel which I use to add a fastpath link between my network and controller. Currently the solution to this is to use VLANs which seem to be working as expected and should be easy to swap back to MPLS when I figure out my problem with that.

Currently the implementation is quite naive and results in a duplicated rule for each port on the data-plane to add the correct tag which directs traffic to the controller. This should be easy to reduce down by adding an extra table which tags the port.

The next step is to integrate this with the existing RouteFlow controller and add support for inter-switch links.




Officially started my PhD this month, 1st April. As such I've switched from focusing on libtrace to software defined networking. With short weeks due to Easter, this report is covering the first two weeks of April.

I've started reading over some previous research of WAND students and other useful resources that Brad has directed me to.

With Brad's assistance I've also setup a virtual machine running a controller, running RouteFlow --- essentially a Vandervecken deployment. This connects to a mininet VM where an Open vSwitch is acting as the OpenFlow controllable device.

I've meet with Richard Nelson and he's suggested adding 'fastpath' to RouteFlow as a first project to get a feel for SDN development and break up the task of researching literature.

Packets that need to be processed by the controller are currently sent via a OpenFlow PACKET-IN message and if necessary returned to the via a PACKET-OUT message. There is a lot of overhead involved in this process, including adding extra meta data to a packet. Fastpath aims to eliminate this by using the data plane to forward packets to the controller quickly by adding OpenFlow rules. One difficulty is attaching the meta-data, i.e. the switch and port number.

I've also meet with Bill Rogers. He wanted to see some good specific use cases for SDN, particularly those which cannot be solved by the current technologies rather than the argument SDN gives people to flexibility to do whatever they want. Also while SDN can make many things easier many problems can be solved by some combination of current technologies.