Last week, I was able to convert routes to OpenFlow rules and install the rules on the switch. The rules installed on the switch now make it possible for hosts on different networks connected to the switch to communicate without packets getting to the gateways of each network.
This week, I'll be working on writing the code to setup fastpath between the OF switch and the Rhea virtual switch. The aim of fastpath is to have a direct link between the OF switch and the virtual switch and have certain types of packets like ICMP being forwarded directly to the virtual switch mapped ports rather being sent to the controller which then examines the packet and sends it to the virtual switch.
Finished adding the ability to set DSCP bits for all the amplet tests
individually as well as globally. Slightly tidied up the way the global
options are turned into individual test options now that there are a few
more of them.
Tidied up the management connections to try to reuse the existing SSL
connection that started the server, rather than always expecting a
separate connection (as is sometimes the case when run standalone). As
part of this, added SSL support to the standalone tests, so now they can
be run standalone with/without SSL, or using it to connect to a normal
Reworked the way watchdogs worked to make sure they will properly
monitor new server threads, or remotely scheduled tests. The central
watchdog management has now been replaced by a timer inside each
server/test process that will ensure the test completes on time.
Worked on the next MBIE funding proposal document. Still got a fair way to go so this will probably eat up a lot of next week too.
Continued trying to identify the remaining Unknown applications in the Waikato Sept 15 traces. Only managed to identify one new protocol (Xunlei Accelerated) but this did account for 14G of unknown traffic on TCP port 8080 so that has gotten rid of the biggest outstanding quantity of unknown traffic. The rest are looking like they might get the better of me -- it's almost all Chinese in origin and I can identify the parent company (Tencent, CERNET, Taobao etc) but actually figuring out which of the myriad of apps these companies own is mostly just trial and error at this stage.
Received feedback for the paper, I've worked a little bit on some of this.
Filled out my PhD progress report. Had a talk with Richard about this he is concerned that my proposed approach might take to long before I start tackling the key issues. As such this is likely to be discussed more next meeting.
Styled the patch manager I've been working on with bootstrap and added some basic documentation ready and other tidying. It now has a name OFCupid and is available on github https://github.com/wandsdn/OFCupid.
Had some interest in libtrace with the DPDK 2.2 library (currently support up to 1.8), it seems that they are working on updating the code to support this.
I'm away for the next three weeks.
Decided to start doing systemd scripts properly and wrote a service file
for the amplet client. Also slightly tweaked the debhelper scripts that
are run to make sure the client doesn't start without configuration and
end up reporting errors to systemd. Had to officially split the Debian
directories now for Wheezy and Jessie as they are starting to diverge
Started work on adding the ability to set the differentiated services
bits in the IP header for all of the AMP tests. This can be set at a
global level, or on a per test basis. So far only the icmp test will
obey the setting, I'll update the rest of the tests next week.
Spent some time trying to remove an unnecessary extra control connection
for tests involving servers started by a remote amplet client. It looks
like I should be able to reuse the connection used to start the server
as the ongoing control channel, but I'm not quite sure how to make this
work best with standalone tests (that expect the server to already be
running, and don't currently encrypt anything). I should be able to tell
if I have a secure control connection or not and take the appropriate
actions, but a bit more planning is required.
I've been working on getting routes converted to OpenFlows rules and installed on the switches, I've been able to establish connectivity i.e send and receive ICMP and ARP packets between the interface on the virtual switch (dp0) that is mapped to a port on the OpenFlow switch. The interface on the virtual switch is the gateway/next-hop for the host(s)/network connected to the mapped port on the OpenFlow switch and it is important that connectivity is established between them for the rest to work.
I intend to establish connectivity between two hosts on a different network, connected to different ports on the OpenFlow switch before this week runs out and I would get started on coding up fastpath.
I got through the first week teaching: mostly just introduction stuff
and easy first year lectures, though I did complete the first COMP312
lab session, which went pretty well considering the volume of students
in the lab. Gave feedback on reports to a few students, which will
require multiple rounds.
The goal with spoofer this week was to get reviews of the sharable tests
from operators. Ryan worked on the page to display only sharable tests,
anonymised, and put it on the spoofer website, currently unlinked. kc
sent it to a mailing list (I helped draft the email) and the only
response over the weekend (that I was aware of) was the sampling bias
issue. I guess the page will be publicly linked shortly. The next
highest priority for Ryan is the protobuf work for the client/server
system. Participated in DHS call with Dan, did some work on the spoofer
subcontract. I need to write up the deliverable document for DHS and
pass a copy past the DITL people.
I worked on the traceroute spoofer stuff again, writing a C parser for
the raw data to get all the raw data I need fast. The perl json stuff
was too slow. I want to put the inferences up on spoofer.caida.org at
some point soon to get public feedback. But after April 1st? And of
course write the paper for IMC to get it off my plate.
Handed off remote controller for bismark to Alex G. Participated in
bismark call with Guilherme and Feamster. Short call as calls go.
Guilherme pointed the bismark units at beamer, where Alex had a
sc_remoted process. No issues on the bismark side but I need to rework
parts of the control socket protocol.
Gave Stephen Marcisak (NPS) lengthy feedback on his Masters thesis that
I am a reader on, on uptime. Good stuff so far. Rob pointed me at
additional raw graphs he has, which are compelling.
Next week: spoofer, teaching, proposal writing.
Focusing on multi-table pipelines this week. I've started by creating a passthrough OpenFlow controller allowing me to intercept and rewrite messages. For this I used the familiar libfluid library, while this works it may not be ideal due to its locking of both read and write of a connection at the same time. This can deadlock when both connections receive a message at the same time and then try to write to the other connection with is locked due to the receive. Currently I spawn a thread per message to work around this, however this is expensive.
I've also started looking at the Pica's multitable pipeline, with can include the MAC matching table and Routing on top of the standard ACL table (where OpenFlow rules are typically installed). These tables can be placed at either a higher or lower priority than the ACL.
In order to test these tables functionality I decided to focus on the MAC table with a modified versions of ryu's simple switch. In order to test this I setup mininet with two bridged networks (in the same subnet), each of which is connected to one port on the Pica. Off each bridged network any number of hosts can be created --- with all packets that traverse between these networks being switched by the Pica.
Ryu's original version of simple switch assumed a single host per port, however in the mininet configuration there are multiple. This in consideration with the matches and actions available in the MAC table lead me to a solution to match ARP packets at a high priority and send these to the controller, and install eth_dst rules beneath that.
I found MAC table flows had to strictly match an eth_dst and vlan_vid as documented, however could optionally match ethertype also. I also found a possible bug, where installing a rule could with send to controller would replay the last packet in, in the case if simple switch this resulted in attempting to install that same rule again and resulted in a infinite cycle (more investigation is needed).
I've started initial thinking and working towards translating these rules from the single table exposed by simple switch into multiple tables. I created a sample ryu implementation that uses multiple tables. And have started working through processing the matches to detect cases where a translation could be made in the passthrough controller.
At the start of the week I, finally, sent a draft of the slow-path benchmarks paper to my supervisors and spent a couple of hours doing a little bit of tidy up.
Continued working away at the Unknown traffic from my libprotoident port study. Added new protocols for Telegram Messenger and Kuguo, as well as improved DNS (especially TCP DNS) and NTP matching. I still have a bit more Unknown traffic to identify before I'd be comfortable putting the results in a paper, but we're getting closer.
Gave my 513 lectures this week. Looking forward to seeing how the class get on with my assignment.
Met with Ryan Jones who is doing an Honours project that will use netevmon to try and find events in the CSC data. Gave him access to the code and a few hints to start out, but I imagine I'll have to dedicate some more time to this over the course of the year.
Fixed this week:
- Working with ITS we have solved the issue where some Eduroam devices
can't communicate with 250-net. The issue was caused by a dangling VLAN
interface in the FCMS-vdom inside the Fortinet firewall that was
blackholing access to our network from some UoW subnets.
- Rolled out openssl patch roll up to various machines to fix DROWN etc.
- Rolled out new rabbitmq to AMP fleet to patch CVE-2015-8786.
- Rolled latest drupal patches.
Business as usual:
Tested Debian upgrade path on warlock on a filesystem snapshot. Still
have a bunch of work to go on wand website to make it work on newer
versions of PHP provided by newer Debian.
Met new honours students, spent time discussing direction of their
Released new version of Bearwall with official systemd support which
means it now works with new ubuntu that will be running on new wand
desktops. Spent some time writing firewall rules to work with docker.
Started conversation with new switch vendor to buy a new multitable
openflow switch. They seem positive to be working with us.
In process of working with another vendor to fix CPU load issues we
are having with their product.
Looked after both Jamie and Perry this week who were in town visiting.