Spent the week working on parallelising the docker build system and adding support for armhf. I now have a script that will spin up a docker instance for each flavour of Debian/Ubuntu, download and install build dependencies, build the appropriate packages and copy them back into the host system.
I got standalone cross-compilation of armhf packages working within a single instance, but appear to have missed something when adding it to the rest of the build scripts - something is looking in the wrong paths for the dependencies. In the mean time it works fine inside the emulator, though is a little slow.
Short 2 day week this week before heading down to Wellington for SDNcon.
I added the 'all' group support so that we can handle the multicast case in ofdpa. I spent the remaining time writing tests for the satisfy match part of the TTP code. This included the checking for correct processing of the 'meta' elements within the lists (which are used throughout TTP's).
For SDNcon Brad, Kit, Michael and I teamed up implement faucet, a VLAN switch, for the ofdpa pipeline. We were successful in doing so, the code is now up here https://github.com/wandsdn/faucet.
SDNcon was great to meet up with other researchers. I had a chance to meet students from Victoria and UNSW. Victoria also has students working on ofdpa with an AS4600, like us, and I was able to get some newer firmware from them - I'm still to test this. It seems that everyone is having similar issues. I look forward to keeping in touch and sharing knowledge with everyone.
Continued reading over Stephen's thesis.
Further refined my event dashboard improvements. Added an algorithm that should recognise redundant event groups based on ASNs that the groups have in common with other groups that occur at the same time. This allows us to get rid of a large number of the vague UoW-REANNZ-AARNet, REANNZ-AARNet and UoW-REANNZ groups that were cluttering up the dashboard on prophet. Found and fixed a few bugs with the self-updating dashboard that were causing event groups to disappear or appear in the wrong order.
Added a working summary graph to the traceroute path map view, with the added benefit of making the selector appear and actually work for this graph.
Continued to battle with InfluxDB's memory usage on prophet. Experimented with tuning a variety of configuration options to try and avoid some of the surges that we occasionally see. Since these surges usually eventually result in the OOM killer being invoked, we need to be able to better control the memory usage before we can consider rolling InfluxDB into production.
Tidied up a couple of issues encountered while installing Wheezy packages for the amp-web server components - eventing schema is now created at install time rather than run time and permissions are set correctly so that events can be fetched by the web user. Fixed the AS name parsing code to always return something sensible, after discovering that our data source had messed up many of the AS name fields, so we were using incomplete data.
Updated some of the server install instructions to match changes to packages and dependencies. Fixed the rabbitmq installation instructions to use their new keys.
Integrated the access control list with the control socket processing code, so now all connections are checked for valid permissions (start test servers, run tests remotely). I've got slightly stuck on making sure that failures are reported back properly rather than just failing, as it gets convoluted when a remotely started test needs to start a server elsewhere but lacks the server permissions.
Still working towards the goal of generating and installing some rules into the ofdpa pipeline. I've continued with processing groups in the TTP, this is needed in the of-dpa pipeline as groups are needed to install output actions. I've also found that in some other cases other actions need to be applied in the group too rather than in the instructions. One such that is seen is the pop_vlan, which is done in the group rather then the instruction. As such I now walk into the groups and their buckets and process the actions included. However this is still not catching all cases, I still need to implement multicast and broadcast, which in ofdpa is control by a group with group_type="all" in which its buckets reference indirect groups which output to a port. This case is interesting because it is a copy operation to each indirect group, where each indirect group can separately choose if it pops a vlan or not.
An updated version of the of-dpa TTP pipeline has come out this week, it looks like they've added one or two fields, however there are no code and or documentation updates accompanying it. I've spent some time working through fixing up the table type pattern (a script of regex replacements which works with the new and old versions), I'm now fully parsing all the identifiers and groups that have been defined and have corrected a number of typos. I hope to push these changes back to the github soon.
Spent most of my week looking into methods for reducing some of the redundant event groups that appear on the amp-web dashboard. Came up with an algorithm for detecting smaller groups that are already covered by one large group, as well as one for detecting when a large group should be removed in favour of the smaller sub-groups.
Implemented my techniques on prophet, but the range of event groups that I get are a bit limited to be sure that everything is working correctly. Next week I may look into grabbing a copy of skeptic's event database to see how well things work on a more diverse set of event groups.
Spent some time reading over Stephen's revised thesis.
Used the new docker build system to build Wheezy packages for the AMP server components. Haven't got it working in parallel, but it is still very useful to build in a clean environment to make sure dependencies are correctly recorded. Updated skeptic to use the new packages and ran into some issues (mostly permissions and non-packaged dependency related) that weren't encountered during previous rollouts on Jessie or test runs in a VM.
Spent some time finishing up the access control for amplet control connections and writing some basic unit tests. The lists all perform as expected, with more specific rules overriding less specific ones. The next step will be to integrate it into the control socket processing so that it gets checked when clients connect and issue commands.
Working towards a short term goal of trying to install automatically generated flows in the ofdpa pipeline, before tackling the larger issue of dependencies between rules. This requires me to install entire Flows match+instructions/actions, as such I've been working on adding instruction and action matching, previously I've only been considering the match portion.
I have been working through the process of parsing in instructions and actions much like I have already done for the match portions for both the TTP and converting ryu flows to a intermediary representation. As part of parsing the action I've created a simple topological sort which returns a normalised representation for apply actions, as this action list is applied to the packet in the order specified, however only some actions are dependent on earlier ones. I've restructured some common code parsing meta items in lists into a TTPList type which handles the parsing of meta items and the logic around satisfying the condition imposed.
For now apply actions are only be matched with apply and write with write, however both will be considered in the future.
The next step is parsing groups, which also include instructions in their buckets!! and are required to output packets in the ofdpa pipeline.
Back into it after a couple of weeks spent moving house.
Worked with Brendon to get nntsc, ampy and amp-web upgraded on skeptic. Also got netevmon running on skeptic so we now have event detection running on the public AMP mesh.
While I was away, InfluxDB ran out of memory and died on prophet. Trying to catch up on the backlog of data kept causing InfluxDB to use ridiculous amounts of memory so I had to spend a decent chunk of my week chasing the cause down. At this point, my biggest wish is that someone will add sensible memory management to InfluxDB.
Did a bit of preliminary writing of a possible paper on NNTSC. Organised some of my thoughts on network measurement ecosystems and turned them into a blog post.
In the past couple of weeks I've been working further on processing Table Type Patterns and matching rules into these. In terms of processing Table Type Patterns I have extended the fitting of existing flow rules into the pipeline to consider which prior tables a packet is required to traverse in order to get to a table which a rule will fit. Then given that path check that flow rules that will match the required packets can in fact be installed. All of this is currently only considering the match component.
I've also been working on the task of splitting a single table of rules in to multiple tables. This process involves finding which parts of a flow rules accurately predicts its actions, i.e. finding an ideal place a set of rules can be split across tables. This could be useful in order to fit a restricted pipeline, or find further optimisations (Note in an unrestricted pipeline lower priority rules can trivially be place in a later table with a goto linking the tables). One such optimisation I'm working on is reversing the Cartesian product, as is seen in switching any source mac can go to any dst mac, resulting in a n^2 expansion when placed into a single table. As such I try to detect this case once flows have been split and condense down these rules. To get this working well I've found adding fake rules that go out of the in_port help, as openflow will drop these packets and have attempted normalising cases such as when vlans are present such that all untagged packets are promoted to include vlans resulting in more rules overlapping.
With both the processing of Table Type Patterns and Splitting of flow rules I've been working on some basic unit tests. I've found unit tests useful as this code is starting to grow in size to detect bugs etc.
Last week with Brad found a newer version of ofdpa for the Accton and with his assistance we have successfully installed this. I've updated my simplest switch to work on the newer pipeline and install some functional rules. So far I've found somewhat of a version mismatch between ofdpa and indigo agent missing some match fields required to install certain types of rules, however in the case of simplest switch I've worked around this. I've emailed Accton to see if it is possible to get an updated version. Despite this, the version of ofdpa seems very functional and I'm hoping to be able to do most things I want on it.